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ABSTRACT 


The  Dq>aitmait  of  Defense  expends  billions  of  dollars  on  software  development  and 
maintenance  annually.  Many  Depaitment  of  Defense  projects  fiul  to  be  coiiq)leted,  at 
large  naonetary  cost  to  the  government,  due  to  the  inability  of  current  software 
cost>estimation  tedmiques  to  estimate,  at  an  early  project  stage,  the  level  of  effort 
required  for  a  project  to  be  coiiq)leted.  One  reason  is  that  current  software 
cost-estinution  models  tend  to  perform  poorly  when  applied  outside  of  narrowfy-defined 
domains. 

Machine  learning  offers  an  ahemative  approach  to  the  current  models.  In  nuichine 
learning,  the  domain  ^ecific  data  and  the  coioputer  can  be  coupled  to  create  an  engme  for 
knoAdedge  discovery.  Using  neural  networks,  genetic  algorithms,  and  genetic 
programming  along  whb  a  publidied  software  project  data  set.  several  cost  estimation 
models  were  developed.  Testing  was  conduaed  udng  a  separate  data  set.  All  three 
tedmiques  diowed  levels  of  performance  that  indicate  that  each  of  these  tedmiques  can 
provide  software  project  managers  with  capabilities  that  can  be  used  to  obtain  better 
software  cost  estimates. 
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I.  INTRODUCTION 


A.  BACKGROUND 

The  use  of  conq>uters  and  coi!q>utiDg  tedmology  within  the  Dqpaitment  of  Defense 
continues  to  grow  at  an  accelerating  rate.  As  the  use  of  computers  has  e^anded,  so  has 
the  need  for  computer  software.  While  con:g)uter  hardware  has  decreased  in  cost  relative 
to  its  performance,  the  cost  of  software  continues  to  increase.  It  is  estimated  that  the 
Department  of  Defense  ^ends  approximately  thirty  billion  dollars  aimually  in  the 
acquisition  and  maintenance  of  software  (Boehm,  1987,  pg.  43).  Virtually  no  area  within 
the  military  has  escaped  the  "software  invagon."  From  the  million-plus  lines  of  code  for 
the  Seawolf  submarine's  BSY-2  computer  system  to  the  two  million  lines  of  code  in  the 
Navy's  NALCOMIS  Phase  n  logistics  system,  software  is  now  one  of  the  driving  &ctors 
in  the  success  of  the  United  States  military. 

Although  the  technology  used  to  develop  software  has  improved  throu^  the  use  of 
such  methods  as  object-oriented  programming  and  computer-aided  software  engineering 
(CASE) ,  one  software  management  area  which  remains  underdeveloped  is  software  cost 
estimation.  Since  the  greatest  expense  in  a  software  development  effort  is  manpower, 
software  cost  estmation  models  focus  on  estimating  the  effort  required  to  complete  a 
particular  project.  This  esdmate  of  effort  required  can  then  be  trandated  into  dollars  u^g 
the  appropriate  labor  rates.  The  current  cost  estimation  models  available  to  software 
project  managers  fail  to  provide  sound  estimates  in  a  consistent  marmer,  even  within  their 
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own  narrowly  defined  domains.  In:q)roper  estimation  of  costs  is  a  major  reason  why  many 
Dq)aitment  of  Defense  software  projeas  have  fiiUed.  With  a  decreasing  budget  for 
defense,  it  is  of  even  greater  necessity  that  managers  have  available  to  them  effective 
software  cost  estimation  tools. 

There  are  many  models  available  for  software  cost  estimation.  One  of  the  more 
popular  models  is  the  Constructive  Cost  Model,  or  COCOMO,  developed  by  Barry 
Boehm  t^e  at  TRW  (Boehm,  1981,  pg.  493).  This  model  is  based  on  a  database  of 
sixty-three  projects  developed  at  TRW  during  the  1960's  and  1970's  and  is  descnbed  in 
detail  in  Boehm's  book,  Software  Engineering  Economics.  Part  of  the  reason  for  the 
popularity  of  COCOMO  is  that  it  is*  relatively  easy  to  apply  and  it  resides  in  the  public 
domain.  Other  models,  such  as  ESTBVIACS,  developed  by  Howard  Rubin  and  currently 
the  property  of  Computer  Associates,  Inc.,  are  proprietary  due  to  the  nature  of  the  project 
data  from  which  they  were  developed.  A  common  thread  that  exists  in  aU  of  these  models 
is  that  they  were  developed  by  domain  everts  and  are  heavily  dependent  upon  the 
judgement  of  the  e^q^ert  for  the  determination  of  the  model  inputs  and  relationsh^s 
extracted  from  the  software  project  data. 

An  akemative  approach  to  model  development  that  can  reduce  the  need  for  the 
domain  e>qpert  to  act  solely  on  judgement  is  throu^  the  use  of  artificial  inteUigoice  (AI), 
^edfically  the  use  of  machine  learning.  While  artificial  intelligence  has  been  a  subject  of 
study  smce  the  1950's,  it  has  periodically  been  greeted  with  skepticism  for  a  perceived 
inability  to  deliver  at  the  level  of  performance  promised  by  hs  proponents,  fri  recent  years. 
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as  both  computer  hardware  and  software  have  pown  more  powerful  and  the  objectives  of 
AI  have  been  better  defined,  the  field  of  artificial  intelligence  has  seen  a  resurgence. 
Teduiiques  are  now  available  that  have  the  ability  to  provide  solid  resuhs  over  a  range  of 
problems  when  appropriately  applied.  An  area  of  great  activity  today  in  artificial 
intelligence  is  machine  learning.  Knowledge  acquisition  and  classification  is  a  very  labor 
intensive  task.  The  conqruter,  with  its  ability  to  toO  without  time  oflT  for  vacations  or 
holidays,  provides  a  natural  platform  for  the  automation  of  the  knowledge  acquisition 
process,  or  to  at  least  serve  as  an  assistant  in  the  knowledge  acquisition  process.  The 
application  of  machine  learning  to  the  problem  of  software  cost  estimation  is  the  focus  of 
this  investigation. 

B.  OBJECTIVES 

Current  methodologies  for  software  cost  estimation  vary  widely  in  their  estimates 
when  presented  with  identical  inputs  (Kemmerer,  1987,  pg.  416).  A  common 
characteristic  of  most  cost  estimation  models  is  that  they  tend  to  provide  only  marginally 
useful  resuhs  within  a  rigid  domain  that  is  often  too  narrow  to  be  of  use  when  pursuing 
new  types  of  software  development  projects.  The  enqrhaas  of  this  thesis  is  to  develop  a 
methodology  for  the  application  of  machine  learning  techniques  to  software  cost 
estimation.  The  three  machine  learning  techniques  cb  -tsen  for  this  project  are  nemal 
networks,  genetic  algorithms,  and  genetic  programming.  The  performance  of  the  models 
derived  fi’om  eadi  machine-learning  technique  are  evaluated  and  conqrared  with  reject  to 
ease  of  application,  accuracy,  and  extendbilhy.  Additionally,  the  models  are  analyzed  to 


see  msigbt  they  can  provide  into  the  relative  inq>ortance  of  the  available  input 
variables. 

C.  THE  RESEARCH  QUESTION 

The  primary  research  question  of  this  thesis  is  to  determine  the  fea^Qity  of  applying 
machine  learning  to  the  problem  of  software  cost  estimation.  This  research  uses  the 
COCOMO  data  set  for  model  training  and  the  COCOMO  and  Kemmerer  data  sets  for 
testing  the  resulting  models.  The  other  inqronant  questions  of  this  thesis  are  to  determine 
the  effectiveness  of  machine-learning  techniques  when  applied  to  the  cost  estimation 
problem  and  what  insight,  if  any,  can  be  gained  from  the  machine  generated  models  into 
cost-estimation. 

D.  SCOPE 

Tbe  scope  of  this  thesis  is  to  apply  the  three  machine  learning  techniques  to  the  cost 
estimation  problem  using  the  COCOMO  and  Kemmerer  data  sets  and  to  determine  the 
extent  to  vvdrich  they  are  appropriate  and  inagbtful  to  the  cost  estimation  process.  The 
scope  of  this  theas  is  not  to  develop  to  develop  a  better  model  for  cost  estimation  but 
instead  to  develop  a  methodology  for  the  application  of  machine  learning  to  this  problem. 

E.  METHODOLOGY 

This  thesis  is  divided  into  four  steps.  First,  the  sixty-three  projects  in  the  COCOMO 
data  set  are  analyzed  and  partitioned  into  training  and  testing  data  sets.  Secondly,  the 
three  machine  learning  techniques  are  apph'ed  using  the  training  data  set  as  the  means  for 
knowledge  acquisition.  The  resulting  models  are  then  tested  u^g  the  COCOMO  data  not 


4 


mchided  in  the  training  set.  A  variety  of  parameters  and  constraints  ^ecific  to  each 
tedinique  are  varied.  In  a  second  iteration  of  this  process,  all  of  the  63  COCOMO 
projects  are  used  as  the  training  set,  and  a  second  set  of  15  projects,  identical  to  the  set 
used  by  Kemmerer  in  his  1987  article  in  the  Communications  of  the  ACM  is  used  as  the 
testing  set.  The  goal  of  this  sequence  of  training  and  testing  is  to  see  how  well  the  various 
machine  generated  models  can  perform  across  different  software  development  domains. 
Since  the  fifteen  projects  presented  by  Kemmerer  were  developed  outside  of  TRW,  wiiich 
is  u^ere  Boehm's  data  was  obtained,  the  capability  of  the  models  to  generalize  can  be 
tested. 

In  the  case  of  neural  networks,  muhiple  network  configurations  are  tested.  The 
resuhing  trained  neural  networks  are  ranked  on  overall  performance  against  actual  projea 
effort  and  against  each  other. 

In  the  third  step  of  this  thesis,  a  goietic  algorithm  is  applied  to  the  training  data  u'ing 
the  original  CCX^OMO  model  structure  as  a  fitness  fimction  template  in  order  to  obtain  a 
revised  COCOMO  parameter  set.  These  new  values  will  be  used  to  evaluate  the  testing 
data  using  the  Intermediate  COCOMO  model  structure.  Initially,  the  genetic  algorithm  is 
relatively  unconstrained.  In  fiirther  tests,  constraints  are  added  to  the  genetic  algorithm 
fitness  fimction  to  determine  the  effect  on  the  resulting  model  parameters  and  on  the 
model's  accuracy  with  reject  to  the  testing  data. 

Finally,  the  relatively  new  concept  of  gaietic  programming  is  applied  to  the  training 
data  set  to  see  what  algorithmic  models  for  cost  estimation  can  be  derived  purely  fi-om  the 
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training  data  with  no  preconceived  functional  form  or  structure  provided.  The  resuhing 
models  will  again  be  tested  using  the  test  data  set. 

The  resuhing  models  from  ail  techniques  will  also  be  analyzed  to  see  v^at  insights,  if 
any,  they  provide  into  the  overall  cost  estimation  process.  The  way  that  the  various  input 
variables  are  used  by  the  machine  generated  models  may  provide  an  opportunity  to 
discover  relationsh^s  previously  unnoticed.  This  is  an  additional  benefit  of  using  the 
conqmter  in  the  knowledge  acquishion  process  and  h  is  explored. 

F.  ORGANIZATION  OF  STUDY 

Chapter  I  provides  a  general  backgroimd  for  this  study.  Chapter  U  discusses  the 
CCK^OMO  model  and  the  three  machine  learning  techniques  in  greater  detail,  including 
some  history  and  the  basic  princ^les  of  each.  Chapter  III  describes  the  experimentation 
process  and  setup.  Chapter  IV  discusses  the  results  of  the  various  experiments  and 
compares  these  resuhs  with  those  obtained  using  the  original  COCOMO  model  in  its 
various  forms.  Additional  insights  into  the  cost  estimation  question  will  be  noted  and 
analyzed  at  this  time.  Chapter  V  is  the  conclusions  and  recommeLdations  resulting  from 
this  inquiry. 
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n.  MODELS  AND  MACHINE  LEARNING  PARADIGMS 

A.  THE  CONSTRUCTIVE  COST  MODEL 

1.  Development  History 

The  Constructive  Cost  Model,  commonly  referred  to  ss  COCOMO,  was 
developed  by  Barry^  Bodun  in  the  late  1970's,  during  his  tenure  at  TRW,  and  published  in 
1981  in  his  text,  Software  Engineering  Economics.  This  model,  v^hich  is  actually  a 
hierarchy  of  three  models  of  increasing  detail,  is  based  on  a  study  of  sixty-three  projeas 
developed  at  TRW  from  the  period  of  1964  to  1979.  In  his  text,  Boehm  describes  the 
development  of  COCOMO  as  being  the  resuh  of  a  review  of  then  available  cost  models 
coupled  with  a  De^hi  exercise  that  resuhed  in  the  origmal  model  This  model  was 
calibrated  using  a  database  of  12  completed  projects. 

When  that  model  failed  to  provide  a  reasonable  e>q>lanation  of  project  variations 
^en  e>q)anded  to  a  56  project  database,  the  concept  of  multiple  developmoit  inodes  was 
added.  Three  development  modes  were  defined  as  organic,  semidetadhed,  and  einbedded. 
Organic  mode  refers  to  relatively  small  (<  SO  KDSI)  stand-alone  projects  with 
non-rigorous  ^ecifications  that  typically  use  small  development  teams  and  involve  low 
lisL  Organic  mode  projects  are  usually  thou^t  to  have  higher  levels  of  productivity  than 
the  other  two  modes  due  to  their  nnall  size  and  flexibility  in  q>ecifications.  Semidetadied 
mode  refers  to  projects  of  small  to  medium  aze  (iq>  to  300  KDSI)  that  involve 
diaracteristics  of  both  organic  and  embedded  projects,  sudi  as  a  system  that  has  some 
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rigorous  specifications  and  some  non-rigorous  q>ecifications.  Embedded  mode  refers  to 
projects  of  all  sizes  that  typically  have  rigorous,  non-negotiable  ^ecifications  that  are 
tii^tty  coupled  to  either  hardware,  regulations,  operational  procedures,  or  a  combination 
of  these  factors.  Einbedded  mode  projeas  generally  require  innovative  architectures  and 
algorithms  and  entail  greater  risk  than  organic  or  semidetached  projects  of  similar  size. 
(Boehm,  1981,  pp.  76-77) 

Once  the  three  development  modes  were  defined,  they  were  calibrated  using  the 
origmal  56  project  database  to  provide  greater  accuracy.  Seven  more  projects  were  later 
added  to  the  project  database  for  a  total  of  sixty-three.  Appendix  A  contains  the  entire 
database.  Boehm  describes  COCOMO  as  not  bemg  heavily  dependent  on  statistical 
analysis  for  calibration  due  to  the  inherently  conq)lex  nature  of  software  development. 
Instead,  COCOMO  relies  on  enqiirically  derived  relationships  among  the  various  cost 
drivers.  These  cost  drivers  are  related  to  attributes  associated  with  the  product  being 
developed,  the  target  conq)uter  platform,  the  development  personnel,  and  the  developmoit 
environment.  (Boehm,  1981,  pg.  493) 

The  COCOMO  model  is  popular  since  it  is  easy  for  managers  to  apply  and  h  is 
widely  taught  in  software  management  courses.  In  his  text,  Bodim  provides  clear 
d^nitions  of  the  model  inputs  through  a  variety  of  tables  and  diarts.  This  type  of 
presatation  allows  managers  to  understand  what  costs  the  model  is  estimating  and  how 
the  estimates  are  readied.  Therefore,  the  model  can  also  be  used  by  managers  to  poform 
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sensitivity  analyses  to  examine  tradeofis  on  a  variety  of  differoit  software  developnoent 
issues.  (Boehm,  1984,  pg  13) 


2.  Basic  COCOMO 

Baac  COCOMO  is  the  an^lea  version  of  the  model  It  is  designed  to  provide 
a  macro  level  scaling  of  project  effort  based  on  the  mode  of  development  and  the 
projected  size  of  the  project  in  thousands  of  delivered  source  inaructions  (KDSI).  Boehm 
describes  its  accuracy  as  limited  though,  due  to  the  lack  of  factors  to  account  for 
differences  in  project  attributes,  such  as  hardware  conaraints  and  personnel  experience. 
(Boehm,  1981,  pg.  58)  The  three  equations  for  Basic  COCOMO  are; 


Table  2-1  Basic  COCOMO  Effort  Equations 


Mode 

Effort 

Organic 

MM=2.4(KDS1)’“ 

Semidetached 

MM=3.0(KDSI)’” 

Embedded 

MM*3.6(KDSI)'^ 

The  accuracy  of  Basic  COCOMO  is  only  satisfactory.  For  the  axty-three 
projects  in  the  database,  Basic  COCOMO  eaimates  are  within  a  fiictor  of  1.3  of  the 
actuals  jua  29%  of  the  time  and  within  a  factor  of  2  of  the  actuals  jua  60%  of  the  time. 

(Bodun,  1981,  pg.  84) 

3.  Intermediate  COCOMO 

Intermediate  COCOMO  tries  to  inqirove  upon  the  accuracy  of  Baac  COCOMO 
by  introducing  the  concept  of  coa  drivers,  which  act  as  effort  muh^liers.  Fifteen  fiictors 
have  been  identified  by  Bodun  as  attributes  that  affect  the  effort  required  on  a  particular 


project.  These  fifieoi  drivers  are  grouped  in  four  attribute  categories  and  are  shown  in 
Table  2-2. 


Table  2-2  Intermediate  COCOMO  Cost  Drivers 


Pniduct 

Attributes 

Computer 

Attributes 

Personnel  Attributes 

Project  Attributes 

RELY  Required 

Software 

Reliatality 

TIME 

Execution  Time 

Constraint 

ACAP 

Analyst  Capability 

MODP 

Modem  Programming 

Practices 

DATA 

Da^.  Base  Size 

STOR 

Main  Storage 

Constraint 

AEXP 

Applications  Experience 

TOOL 

Use  of  Software  Tools 

CPLX 

Product 

Complexity 

VIRT 

Virtual  Machine 

Volatility 

PCAP 

Programmer  Capability 

SCED 

Required  Development 

Schedule 

TURN 

Computer 

Turnaround  Time 

VEXP 

Virtual  Machine 

Experience 

LEXP 

Programming  Language 

Ex|3erience 

These  fifteen  cost  drivers  are  used  in  conjunction  with  a  set  of  scaling  equations 
amilar  to  those  used  in  Basic  COCOMO.  This  nominal  effort  is  then  modified  by  using 
the  product  sum  of  the  cost  drivers  (defined  as  the  Effort  Adjustment  Factor,  or  EAF)  as 
defined  for  a  particular  project  to  obtain  the  Intermediate  COCOMO  estimate.  The 
nominal  equations  for  the  Intermediate  COCOMO  model  are  shown  in  Table  2-3: 
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Table  Z-3  Intcrniediatc  COCOMO  Nominal  Effort  Equations 


DevdopmeDt  Mode 

Nominal  Effort  Eqaatkw 

Organic 

(MMX«<-3.2(KDSD'“ 

Scmidetachad 

(MM)*^-3.(XKI>SD’“ 

Embedded 

(MMX«,-2.KKDSI)’* 

Ibe  cost  drivers  are  subdivided  into  Iheral  rating  cat^ories  that  run  from  Vmy 
Low  to  Extra  Hi^.  The  method  for  determining  i^ch  rating  category  each  driver  fidU 
into  for  a  qpedfic  project  is  outlined  by  Boehm  in  his  text.  (Bodun,  1981,  pg.  1 19)  Once 
the  rating  category  is  determmed,  the  numerical  value  for  the  cost  driver  is  obtained  from 
atabte.  This  table  can  be  found  in  Appendix  B.  The  Intermediate  COCOMO  estimate  of 
project  effort  is  then  ^en  by  the  equation  (MM)^*(£ffott  Adjustment 

Factor).  The  Intermediate  COCOMO  model  serves  as  one  of  the  bases  of  comparison  for 
the  various  methodologies  presented  in  the  thesis.  Intermediate  COCOMO  is  diosen  for 
comparison  smce  the  project  data  published  in  Boehm's  text  is  only  presented  at  this  level 
ofdetafl. 

4.  DetaUed  COCOMO 

Detaded  COCOMO  is  an  additional  refinement  of  the  noodel  that  allows  the 
effint  estinuition  to  be  fivtLrr  detafled.  This  veraon  of  the  model  attenopts  to  inqrrove  foe 
estimate  by  overconung  two  limits  of  foe  Intermediate  modd.  First,  Detailed  COCOMO 
refines  foe  estimate  introdudng  cost  drivers  that  vary  for  foe  various  devdopment 

ffoases  and,  second,  it  allows  for  foe  distribution  of  various  cost  drivers  over  three  vertical 
levds:  module,  sub^em,  and  Q^stem  Due  to  foe  ruture  of  foe  COCOMO  database  as 
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prescDted  in  Bodun's  text,  the  Detailed  versira  of  the  model  will  not  be  used  as  the  basis 
of  conq)aris(»i  during  the  evahiatioo  of  the  various  madiine  learning  paradigms.  (Bodun, 
1981,  pp.  344-345) 

B.  NEURAL  NETWORKS 

1.  History 

The  neural  network  paradigm  has  a  colorful  history  dating  back  to  the  IPSCs. 
This  paradigm  grew  out  of  the  efforts  of  early  artiSdal  intelligeoce  (AI)  researchers  to 
construct  systems  that  mimicked  the  actions  of  neurons  in  the  human  brain.  In  1958, 
Frank  Rosenblatt  published  a  paper  that  defined  a  neural  network  structure  called  a 
percqrtron  (Eberhart,  1990,  pg.  18).  This  paper  outlined  the  princ^les  that  inibnnation 
could  be  stored  in  the  form  of  connections  and  that  information  stored  in  this  maimer 
could  be  iqidated  and  refined  by  the  addition  of  new  coimections.  This  researdi  laid  the 
foundation  for  both  types  of  training  algorithms,  supervised  and  unsupervised,  that  are 
used  in  neural  networks  today. 

While  work  continued  in  the  neural  networit  field  during  the  1960's  and  1970's 
their  usefulness  was  in  doubt  until  the  publication  of  a  paper  by  John  Hopfield,  firom  the 
CaHfomia  Institute  of  Tedmology  (Eberhart,  1990,  pg.  29).  Hopfield’s  worh  was 
haqioitant  because  he  identified  network  structures  and  algorithms  that  could  be  defined  in 
a  general  nature  and  he  was  the  first  to  identify  that  networks  could  be  inqilemented  in 
dectronic  drcuhry,  whidi  interested  semiconductor  manufiicturers.  The  publication  in 
1986  ot  Parallel  Disirilmted  Processing  by  the  Parallel  Distributed  Processmg  Researdi 
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Groiq)  ensured  the  rebirth  of  neural  networks  by  describing  in  great  detail  a  variety  of 
architectures,  attributes  and  transfer  fiinctions  (Ebeihart,  1990,  pg.  32).  Since  then,  the 
variety  and  application  of  neural  networks  have  grown  immoisely.  The  Defense 
Advanced  Researdi  Projects  Agency  ^onsored  a  neural  network  review  in  1988  and 
published  a  rq)ort  on  the  field  (Maren,  1990,  pg.  20).  This  publication  tended  to  validate 
the  field  as  being  woithy  of  research  fiinding  and  now  there  are  a  variety  of  journals  and 
publications  dedicated  to  this  field,  as  well  as  an  international  society  (Maren,  1990,  pg. 
20). 

2.  Theory 

There  are  a  wide  variety  of  neural  network  models  in  use  today.  One  of  the 
most  common  networks,  and  the  one  chosen  for  use  in  this  the^  is  the  baclq)ropagation 
network.  In  this  case,  backpropagation  refers  to  the  training  method  used  for  this 
network.  The  network  in  operation  acts  in  a  feed-forward  manner.  The  basic  prindple  of 
operation  is  single.  A  backpropagation  network  is  typically  constructed  of  an  input  layer 
of  neurons,  an  output  layer  of  neurons,  and  one  or  more  hidden  layers  of  neurons.  Bias 
neurons  may  also  be  defined  for  each  hidden  layer.  Each  neuron  (or  node)  is  defined  by  a 
transfer  fimetion.  In  the  case  of  the  backpropagation  network,  the  fimetion  usually  has  a 
sigmoid  or  S-shape  that  ranges  asynqitotically  between  zero  and  one.  The  reason  for 
dioo^g  the  sigmoid  is  that  the  fimetion  must  be  continuous  differentiable  and  should  be 
asynq)totic  for  infinitely  large  positive  and  negative  values  of  the  independent  variables. 
(Maroi,  1990,  pg.  93)  The  neurons  in  each  layer  are  then  assigned  a  weighted  connection 
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to  each  neuron  in  the  foDowing  layer.  These  connection  weights  are  established  randomly 
v^on  initialization  of  training  and  then  recalculated  as  the  network  is  presented  with  the 
training  patterns  until  the  error  of  the  output  is  minimized.  The  method  that  adjusts  the 
weights  is  known  as  the  Generalized  Deha  Rule  which  is  a  method  based  on  derivatives 
that  allows  for  the  connection  wei^ts  to  be  adjusted  to  obtain  the  least-mean  square  of 
the  error  b  the  output.  (Maren,  1990,  pg.  99)  Bus  neurons,  if  used,  simply  provide  a 
constant  bput  signal  to  the  neurons  b  a  particular  layer  and  relieve  some  of  the  pressure 
from  the  leambg  process  for  the  connection  weights.  A  simple  network  dugram  is  shown 
b  Figure  2-1. 
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A  bac]q>ropagation  network  is  capable  of  generaUzing  and  feature  detection 
because  it  is  trained  whb  different  exan:q)les  udiose  features  become  embedded  in  the 
weights  of  the  hidden  layer  nodes  (Maren,  1990,  pg.  93).  An  example  of  the  operation  of 
a  neural  networic  is  provided  by  Maren  and  involves  a  neural  network  deagned  to  solve  an 
XOR  classification  problem.  The  diagram  for  this  network  showing  the  configuration  at 
initialization  and  upon  completion  of  training  is  shown  in  Figure  2-2  (Maroi,  1990,  pg. 

97). 

The  exanq>le  given  by  Maren  is  a  version  of  the  backpropagation  networit  in 
wfaidi  each  hidden  layer  neuron  can  have  a  threshold  value  which  is  added  to  sum  of  the 
inputs  to  that  neuron  before  the  application  of  the  agmoid  transfer  fimction.  These 
threshold  values  are  found  umg  the  same  Deha  rule  that  is  used  to  find  the  connection 
weights.  During  the  training  process,  the  connection  weights  (and  threshold  values)  are 
adjusted  u^g  the  following  equation: 

+  a  *  Delta(w,j^j)  -  output_activation_level 

where  w^  stands  for  the  new  and  old  values  of  the  connection  weight  betweoi  node  i  and 
node],  and  a  is  a  constant  that  defines  the  magnitude  of  the  effect  of  Deka  on  the  weight. 
Delta  describes  a  fimction  that  is  proportional  to  the  negative  of  the  derivative  of  the  error 
with  respect  to  the  connection  weight  and  output_activation_level  is  the  output  of  the  jth 
neuron.  This  baclq)ropagation  of  error  methanism  allows  the  weights  at  all  layers  to  be 
adjusted  as  the  training  process  is  performed,  including  any  connections  between  hiddmi 
layer  neurons.  (Maroi,  1990,  pp.  100-101) 


Initulizatira  with  Random 
Weights  and  Threshold  Values 


Convergence  to  Stable 
Weights  and  Thresholds 
After  Training  Interactions 


Initial  Resuhs: 

Resuhs  After  Training 

Input  Output 

Input 

Ou^ut 

(0,1)  0.589 

(0,1) 

(1,0)  0.589 

(1,0) 

(1,1)  0.601 

(1,1) 

Figure  2-2  A  Neural  Network  Example  of  an  XOR  Problem  (Maren,  195H)) 
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1 

Ad  exaiiq>le  of  the  operation  of  a  trained  network  is  seen  by  examining  one  of 
the  input  combinations  in  the  exanple  provided  by  Maren  (1990)  in  Figure  2-2.  Take  the 
pattern  (0,1).  This  pattern  means  that  a  zero  is  the  input  to  the  bottom  left  neuron  of  the 
trained  network  and  a  one  is  the  input  to  the  bottom  right  neuron.  Proceeding  up  to  the 
next  layer,  a  vector  mult4>Hcation  of  the  inputs  and  the  cormection  weights  is  performed  to 
determine  the  inputs  to  the  hidden  layer.  In  this  exanq)le  the  hidden  layer  inputs  are 
(0*(-l  1.62)H  1*10.99) «  10.99  for  the  hidden  neuron  on  the  left  and  (0*12.88)  + 

(1*(-13. 13))  = -13.13  for  the  hidden  neuron  on  the  right.  The  threshold  values  for  these 
hidden  neurons  are  added  to  the  inputs  and  then  the  sigmoid  transfer  function  is  applied. 
For  the  hidden  neuron  on  the  left,  this  means  the  Input  to  the  transfer  function  is 
( 10.99+(-6.06)) »  4.94.  The  activation  value  of  the  input  4.94  when  applied  to  a  signmid 
transfer  fimction  that  ranges  between  zero  and  one  is  approximately  one.  For  the  hidden 
neuron  on  the  right,  the  input  to  the  transfer  function  is  (-13. 13+(*7. 19))  =  -20.32.  The 
activatioD  value  when  -20.32  is  applied  to  the  transfer  function  is  approximately  zero.  The 
hidden  layer  outputs  in  this  exaiiq)le  are  one  for  the  left  hidden  neuron  and  zero  for  the 
right  hidden  neuron.  The  input  to  the  top,  or  output,  neuron  is  calculated  by  taking  the 
product  of  the  hidden  layer  outputs  and  the  connection  weights,  or  (1*13.34)+(0*13. 13)  = 
13.34.  The  threshold  weight  of  the  output  neuron  is  added  to  the  input  fi'om  the  hidden 
layer  to  get  the  input  to  the  transfer  function,  which  in  this  case  is  (13.34-K-6.S6))  =  6.78. 
This  number  whoi  applied  to  the  transfer  function  yields  a  value  close  to  one  (0.946), 
which  is  the  de^ed  answer.  (Maren,  1990,  pp.  61-62) 
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3.  Applications 


Neural  networks  are  currently  being  used  in  a  variety  of  applications.  Some  of 
the  major  uses  are  in  the  areas  of  fihering,  image  and  voice  recognition,  financial  analysts, 
and  forecasting  (Zahedi,  1991,  pg.  27).  Commercial  neural  network  software  is  available 
firom  a  variety  of  vendors.  Specialized  prograimnable  hardware  boards  that  contain 
chqtsets  that  can  mimic  the  operation  of  a  neural  network  are  also  available  for  very 
intensive  neural  network  applications.  The  neural  networks  in  this  thesis  were  develc^ed 
using  a  product  fi’om  Cahfomia  Scientific  Software,  Inc.  called  Brainmaker. 

C.  GENETIC  ALGORITHMS 

1.  History 

Nature's  capability  to  find  solutions  for  conqtlex  problems  through  the  process 
of  natural  selection  has  fascinated  researchers  foi  generations.  The  ability  of  organisms  to 
adapt  to  their  environment  by  akering  their  characteristics  through  mating,  which  provides 
an  opportunity  for  gene  crossover  as  well  as  an  occaaonal  genetic  mutation,  has  proven  to 
be  a  powerful  technique  of  problem  solving.  Genetic  algorithms  were  an  outgrowth  of 
early  work  during  the  19S0's  and  1960's  that  attenq)ted  to  condbine  computer  science  and 
evolution  with  the  hope  of  creating  better  programs.  In  the  mid  1960's,  John  Holland 
developed  the  first  genetic  algorithms  that  could  be  used  to  rq)resent  the  structure  of  a 
computer  program  as  well  as  perform  the  mating,  crossover,  and  the  mutation  processes. 
Since  then,  genetic  algorithms  have  grown  in  popularity  and  have  been  applied  to  a  wide 
variety  of  problems,  especially  in  the  field  of  en^eering  deagn,  where  optimization 
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mvolving  brge  numbers  of  independent  variables  is  a  common  situation.  (Holland,  1992, 

Pg  66) 

2.  Theory 

Genetic  algorithms  are  in  the  simplest  definition  another  method  for  seardi  and 
optimization.  However,  they  differ  from  traditional  methods  such  as  hill-climbing  and 
random  walks  in  at  least  four  ways  (Goldberg,  1989,  pg.  7).  First,  genetic  algorithms  use 
codings  of  the  various  parameters  in  a  function,  not  the  actual  function  parameters 
themselves.  This  coding  usually  consists  of  fixed-length  strings  of  characters  that  are 
patterned  after  chromosomes.  Each  string,  or  chromosome,  then  has  a  fitness  value 
associated  with  it  r^ch  is  a  measure  of  how  well  it  performs  in  terms  of  the  criteria 
defined  for  the  probleia  Second,  genetic  algorithms  perform  highly  parallel  searches 
using  a  population  of  points  vice  a  single  point.  Third,  genetic  algorithms  use  the 
objective  function,  or  actual  payoff  information,  to  determine  fimess  instead  of  derivatives 
or  other  information.  Tins  capability  separates  the  genetic  algorithm  fi'om  those  techniques 
that  require  gradient  information  about  the  function  to  perform  their  seardies.  Instead, 
the  genetic  algorithm  has  the  capability  to  work  whh  the  actual  function  itself  Finally, 
genetic  algorithms  use  probabilistic  rules  to  make  shifts  fi’om  on;;  generation  to  the  next 
instead  of  deterministic  rules .  (Goldberg,  1989,  pp.  7-10) 

The  eariest  way  to  understand  the  working  of  the  genetic  algorithm  is  to  see  h  in 
action.  In  the  following  exanqrle,  smular  to  the  Haniburger  Problem  presented  by  Koza 
(1992,  pg.  18),  the  genetic  algorithm  is  used  to  maximize  a  function.  The  coding  of 
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chanctenstics  in  tins  example  is  done  with  the  binaiy  digits  1  and  0  and  the  fitness  of  each 
mdividual  is  singly  the  decimal  value  of  the  binary  representation.  The  initial  population 
of  the  first  generation,  typically  referred  to  as  Generation  0  in  genetic  algorithins,  is 
randomly  generated  and  shown  in  Table  2*4. 

Table  2-4  Generation  0  of  the  GA 


String# 

String  (X.) 

Fitness  flX,) 

1 

101 

5 

2 

on 

3 

3 

001 

1 

4 

010 

2 

As  can  be  seen  from  Table  2-4,  the  best  individual  in  the  initial  population  has  a 
fitness  of  5,  while  the  worst  individual  has  a  fitness  of  1.  The  population  average  is  2.75. 
After  the  initial  population  is  evaluated,  the  next  step  in  the  genetic  algorithm  is  the 
reproduction  and  crossover  process.  The  method  used  to  determine  which  individuals 
reproduce  is  normally  determined  by  a  method  called  fitness-proportionate  reproduction 
(Koza,  1992,  pg.  21). 

In  fitness  proportionate  reproduction,  candidates  for  reproduction  are  selected 
fi>r  the  mating  pool  by  assigning  each  population  member  a  probability  of  reproduction 
based  upon  eadi  population  member’s  fitness.  Under  this  method,  highly  fit  individuals 
have  hi^er  probabilities  for  reproduction  than  lesser  fit  individuals  and  the  effect  of  this 
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reproduction  operation  is  to  increase  the  average  fitness  of  the  population.  Table  2-5 
shows  the  mating  pool  for  Generation  0  based  on  fitness-propoitionate  reproduction. 


Table  2-5  Generation  0  Mating  Pool  Creation 


Gcaeratioa  0 


Gen  0  Mating  Pool 


String 

Fitness 

Mating  ^obability 

(Fitness/Sum  Fitness) 

Mating 

Pool 

Fitness 

101 

5 

5/11*0.454 

101 

5 

on 

3 

3/11=0.273 

101 

5 

001 

1 

1/11=0.090 

Oil 

3 

010 

2 

2/11=0.182 

010 

2 

Total 

11 

Total 

15 

Best 

5 

Best 

5 

Average 

2.75 

Average 

3.75 

Table  2-S  shows  that  the  average  fitness  of  the  population  has  been  increased 
but  that  the  best-of-generation-individual  fitness  remains  the  same.  The  second  genetic 
process,  crossover,  is  ^at  ahows  new  individuals  to  be  formed  which  may  have  better 
fitness.  When  the  reproduction  operation  is  complete,  the  crossover  operation  is 
performed  by  selecting  two  individuals  using  a  unifonn  random  distribution  fi'om  the 
mating  pool  Selection  through  a  random  distribution  is  posable  smce  menobership  in  the 
mating  pool  is  proportionate  to  fitness  (Koza,  1992,  pg.  25).  The  selected  individuals  are 
separated  into  fi’agments  by  breaking  them  apart  at  a  randomly  selected  interstitial  point. 
The  appropriate  fi’agments  firom  the  parents  are  then  recombmed  to  form  new  individuals 
that  are  then  tested  for  fitness.  Table  2-6  shows  the  crossover  sequence. 


Table  2-6  The  Crossover  Operation 


Rtfent  1 

Parent2 

101 

Oil 

Crossover  Fragment  1(F1) 

Crossover  Fragment  2  (F2) 

1- 

0- 

Renuinder  1  (Rl) 

Remainder  2  (R2) 

-01 

-11 

Offspring  1  (F2-t-RI) 

Offspring  2  (FI+R2) 

001 

111 

When  the  crossover  operation  is  cooqjlete,  two  new  individuals  have  been 
created  and  their  fitness  is  evaluated.  The  mating  pool  members  not  selected  for  crossover 
are  copied  into  the  next  generation.  The  number  of  individuals  in  the  mating  pool  that 
undergo  crossover,  the  crossover  rate,  is  determined  in  advance  by  the  individual  using  the 
genetic  algorithm.  The  mutation  operation,  if  allowed  to  occur  at  all,  also  happens  during 
the  rq)roduction  process.  In  the  mutation  operation,  a  single  bit  is  selected  for 
transformation  based  upon  the  probability  of  mutation,  also  establitiied  in  advance  by  the 
individual  usmg  the  genetic  algorithm.  (Goldberg,  1989,  pg.  14)  The  resuhs  of  the 
rq)roduction  and  crossover  operations  in  the  ample  example  are  tiiown  Table  2-7. 
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Table  2-7  Geon^atioa  1  of  the  Genetic  A^orithm 


String 

Fitness 

Method  of  Production 

001 

1 

Reproduction/Crostover 

111 

7 

ReprodiKtion/Crocsover 

101 

5 

Rqxoduction 

010 

2 

Reproduction 

Best  Fitness 

7 

Average 

3.7S 

As  can  be  seen  from  the  table,  fitness>proportionate  reproduction  combined  with 
the  crossover  operation  has  inq)roved  the  average  fimess  of  the  population  and  the 
best-of-generation  fitness  (in  this  case  reaching  the  global  optimum).  This  capability  to 
seardi  and  optimize  usmg  adaptive  techniques  without  the  requirement  for  an  external 
interlace  to  the  user  is  one  of  the  strengths  of  the  goietic  algorithm. 

The  engine  that  gives  the  genetic  algorithm  its  power  is  called  iiiq)]icit 
paraDeHsm.  This  implicit  parallelism  manifests  itself  in  what  Holland  described  as  the 
Fundamental  Theorem.  (Goldberg,  1989,  pg.  19)  The  Fundamental  Theorem  is  derived 
from  the  concept  of  schemata.  A  population  in  a  genetic  algorithm  consists  of  a  set  of 
strings  conq>osed  of  one's  and  zero's  (a  binary  rqrresentation  scheme  is  coimnonly  used). 

If  the  population  conrists  of  binary  strings,  a  schema  can  be  thought  of  as  a  template  that 
describes  subsets  of  strings  using  the  notation  set  { 1, 0,  *}.  The  asterisk  synabol  signifies 
a  "don't  care"  or  "wild  card"  value.  A  string  with  value  { 1 10}  is  then,  as  an  exarrple,  a 
member  of  schema  {!**}  as  well  as  {'*1*},  {11*}  and  {**0}.  In  fret,  a  particular  string 
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in  •  population  is  a  member  of  2****^^*^  sdiema.  A  schema  can  be  thought  of  as 
representing  certain  characteristics  of  a  candidate  solution  to  the  fitness  fimction.  As 
populations  of  strings  (or  chromosomes)  proceed  fi^om  one  generation  to  the  next  through 
the  process  of  rq)roduction  and  crossover,  individuals  whh  greater  fimess  rise  in  number 
at  the  mq)ense  of  lesser  fit  individuals.  Schemata  bdiave  in  a  similar  manner.  A  sdiema 
(or  characteristic  set)  grows  at  a  rate  proportional  to  the  average  fitness  of  that  particular 
schema  over  the  average  fitness  of  the  population.  A  schema  whose  average  fimess  is 
above  that  of  the  population  average  will  be  represented  in  greater  numbers  in  the 
succeeding  generations.  A  schema  idiose  average  fimess  is  less  than  the  population 
average  will  see  hs  numbers  diminisfa.  In  fact,  above  average  schema  will  see  their 
numbers  increase  in  an  exponential  manner  in  succeeding  populations.  This  effect  of 
simuhaneously  increasing  the  fimess  of  the  population  and  promoting  the  e>q>onential 
growth  of  beneficial  sdiemata  (or  characteristics)  throu^  the  reproduction  operation, 
coupled  with  the  crossover  operation  that  creates  population  diversity  through  an  orderly 
exchange  of  characteristics  is  \^at  gives  the  genetic  algorithm  hs  inqilichly  parallel  nature. 

(Goldberg,  1989,  pp.  29-33) 

3.  Applications 

The  use  of  genetic  algorithms  has  increased  greatly  in  the  past  few  years  as 
people  have  realized  the  benefits  they  bring  to  certain  types  of  problems  that  have  reasted 
solution  by  more  traditional  methods.  In  one  case  genetic  algorithms  were  used  to 
develop  control  medianisms  for  a  model  of  a  conq;)lex  set  of  gas  pqrelines  that  tran^ort 
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natural  gas  across  a  wide  area.  Having  only  various  cong>ressors  and  valves  with  wliidi  to 
control  the  gas  flow,  and  with  a  large  time  lag  between  any  action  and  reaction,  this 
problem  had  no  standard  analytic  solution.  A  genetic  algorithm  was  developed  by  David 
Goldberg  at  the  Univer^  of  Illinois  that  was  capable  of  "learning”  the  control  procedure, 
in  another  case,  Lawrence  Davis  used  genetic  algorithms  to  design  communications 
networks  that  maximized  data  flow  with  a  rnmimum  number  of  switdies  and  transmission 
lines.  General  Electric  is  curratly  using  genetic  algorithms  in  the  design  of  new 
commercial  turbofim  engme  cottq)onents.  Using  a  genetic  algorithm.  General  Electric 
engineers  were  able  to  reduce  the  time  required  to  improve  the  design  of  an  engme  turbine 
from  weeks  to  days.  This  is  notable  ^ce  there  are  at  least  100  variables  involved  in  a 
turbine  design  as  well  as  a  large  number  of  conaraints.  (Holland,  1992,  pg.  72*73) 
Goldberg’s  text  lisXs  additional  uses  of  genetic  algorithms  ranging  from  medical  imaging  to 
biology  to  the  social  sciences  (Goldberg,  1989,  pg.  126).  There  are  currently  several 
conferences,  both  national  and  international,  devoted  to  the  discussion  of  genetic 
algorithms. 

D.  GENETIC  PROGRAMMING 

1.  History 

Genetic  programming  is  an  exciting  new  field  in  conq;>uting  pioneered  by  John 
Koza  of  Stanford  Unrver^  in  the  late  1980's.  Koza,  in  his  text  Genetic  Programming: 
On  the  Programming  of  Computers  by  Means  of  Natural  Selection,  describes  the  core 
concq)t  of  his  tedmique  as  being  the  search  for  a  con:q)uter  program  of  a  given  fitness 
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from  the  ^ace  of  all  possible  con^uter  programs  using  the  tools  of  natural  selection.  This 
q>proach  is  differoit  from  more  traditional  artificial  intelligence  tedmiques  in  that  for  any 
particular  problem  using  traditional  tedmiques,  such  as  neural  networks,  the  goal  is 
usually  to  discover  a  ^eciaUzed  structure  that  provides  a  certain  output  given  certain 
inputs.  Koza  refiames  this  statement  by  saying  what  we  truly  want  to  discover  is  a  certain 
computer  program  that  wifi  produce  the  deared  output  from  a  given  set  of  inputs.  Once 
the  problem  is  refiamed  as  that  of  finding  a  highly  fit  program  within  the  ^ace  of  all 
possible  programs,  the  problem  is  reduced  to  that  of  ^ace  seardi.  (Koza,  1992,  pg.2) 

The  technique  of  genetic  programming  is  not  the  first  atteirqrt  at  using 
corrputers  to  try  to  generate  programs.  Researchers  smce  the  19S0's  have  attenq)ted 
using  various  methods  to  generate  programs  ranging  from  blind  random  search  to  asexual 
rqrroduction  and  mutation.  More  recently,  researchers  in  the  genetic  algorithm  field  have 
attenqrted  to  use  genetic  algorithms  to  generate  programs  by  using  ever  more  ^ecialized 
diromosome  representation  schemes  or  by  udng  a  fecial  type  of  genetic  algorithm 
known  as  a  clasafier  system  to  generate  programs  based  on  if-then  rules  (Koza,  1992,  pp. 
64-66).  Koza  is  the  first  researcher,  however,  to  develop  an  appropriate  representation 
scheme  and  methodology  for  applying  natural  selection  techniques  to  the  problem  of 

program  generation. 

2.  Theory 

The  concept  of  a  liighly  fit"  conq)uter  program  as  a  solution  to  a  particular 
problem  is  disturbing  to  most  people  at  first.  We  are  accustomed  to  the  idea  of  a 
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computer  program  being  either  "right”  or  "wrong"  ^en  apphed  to  a  particular  problem. 
Koza  counters  this  soitiment  and  other  amHar  sentiments  by  saying  that  the  process  of 
natural  selection  is  nehhm^  "right”  nor  "wrong”  but  that  nature  supports  many  different 
approadies  to  the  same  problem,  all  of  \riiich  have  a  certain  "fitness".  The  key  to  nature's 
q>proach,  according  to  Koza,  is  that  form  follows  fitness  (Ko/a,  1992,  pp.  1>7).  Ihe 
requisite  tools  used  in  Koza's  approach  are  a  welKdefined  fimction  representation  scheme, 
a  method  for  generating  an  initial  random  population  of  programs,  a  fitness  measurement 
tedmique,  and  a  procedure  for  applying  genetic  operations  to  the  members  of  the 
population. 

The  representation  scheme  chosen  by  Koza  for  genetic  programming  is  based  on 
a  type  of  structure  known  as  a  synibolic  e^resrion,  or  S-e7q>ression.  Since  this  type  of 
structure  is  a  key  coiiq>onent  of  the  programming  language  LISP,  this  language  was 
chosen  as  the  platform  for  developing  genetic  programming.  It  is  not  a  requirement  to  use 
this  language  for  genetic  programming,  but  it  has  many  features  that  ficilitated  the 
development  of  this  technique.  (Koza,  1992,  pp.  70-71)  An  exanq)le  of  a  LISP 
S-erqtression  is  riiown  in  Figure  2-3. 

As  can  be  seen  from  Figure  2-3,  a  LISP  S-expresaon  is  equivalent  to  the  parse 
tree  for  a  particular  program,  which  in  this  case  is  l*(2+3).  The  abOity  of  LISP  structures 
to  serve  as  both  data  (to  be  manipulated)  and  programs  (to  be  executed)  is  another  of  the 
reasons  USP  was  diosen  as  the  language  for  the  development  of  genetic  programming. 
(Koza,  1992,  pg.  71) 
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figure  2-3  A  LISP  S-eipression 


Id  genetic  programming,  these  LISP  S-e>q>ressions  are  con^osed  of  structures 
that  are  derived  from  a  predefined  fimction  set  and  terminal  set.  The  fimction  set  consists 
of  ^atever  domain-^edfic  fimctions  the  user  feels  are  sufiBcient  to  solve  the  problem 
and  the  terminal  set  is  the  argumoits  that  the  fimctions  are  allowed  to  take  on  (Koa, 
1992,  pg.  80).  For  instance,  the  fimction  set  may  be  conqrrised  of  mathenutical, 
arithmetic,  or  boolean  operators  \^e  the  terminal  set  is  usually  comprised  of  variables 
from  the  problem  domain  and  an  un^ecified  integer  or  floating-pomt  random  constant. 
Since  it  is  inqrossible  to  determine  in  advance  what  the  structures  will  look  like,  it  is 
necessary  to  ensure  in  advance  that  each  fimction  possesses  the  property  of  cloaue  (Koa, 
1992,  pg.  81).  Hie  closure  property  requires  that  the  allowable  arguments  for  any 
fimction  be  well-defined  in  advance  to  prevent  the  fimction  from  taking  illegal  arguments. 
While  this  may  seem  like  an  inqiosmg  task,  it  is  usually  not  a  serious  problem  and  is  asQy 
rectified  by  prior  planning  of  the  fimction  definitions.  As  an  exarrqile,  division  by  zero  is 
undefined.  If  the  division  operator  is  a  member  of  the  fimction  set,  then  by  defining  a 
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specul  dhaston  opo’ator  in  advance  that  handles  this  q)ecial  case,  it  is  possible  to  ensure 
closure  on  this  function.  (Koza,  1992,  pg.  82) 

Once  the  rqtresentation  sdieme  is  defined,  the  initial  program  structures  can  be 
generated  (Koza,  1992,  pg  91).  In  Koza's  method,  the  initial  structures  are  generated 
random^  using  a  combination  of  two  techniques,  "fun"  and  "grow*.  The  root,  or 
beginning,  of  one  of  the  LISP  S-expressions,  or  trees,  is  always  a  function,  since  a  root 
that  conned  of  a  terminal  would  not  be  capable  taking  any  arguments.  Once  the  root 
function  is  selected,  a  number  of  lines,  equivalrat  to  the  number  of  arguments  the  function 
requires,  are  created  which  radiate  away  fi'om  the  function.  The  endpomts  of  these  lines 
are  then  filled  by  a  selection  fi'om  the  combined  set  of  functions  and  terminals.  If  another 
fifflction  is  selected  as  an  endpoint,  the  seleaion  process  continues  onward  until  the 
endpoints  consist  of  termmals.  There  is  a  preset  maximum  depth  that  trees  may  attain. 

The  terms  "fun”  and  "grow"  as  mentioned  above  refer  to  the  depth  of  the  trees.  In  the 
"full"  method,  aU  trees  fiUed  until  they  are  at  the  ^ecified  maxinnim  depth.  In  the  "grow" 
method,  the  trees  are  of  variable  depth.  In  practice,  Koza  recommends  an  approach  called 
"ran:q>ed  half-and-halT  that  combines  these  two  approaches  (Koza,  1992,  pp.  91  >92). 
Koza  additionally  recommends  that  each  structure  generated  for  the  initial  random 
population  be  checked  for  uniqueness  to  ensure  a  range  of  diveraty  of  genetic  material 
(Koza,  1992,  pg.  93).  Figure  2>4  gives  a  graphical  rqrresentation  of  the  creation  of  an 
individual  in  the  initial  population. 
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Figure  2-4  Graphical  Representation  of  Tree  Generation 
Once  the  initial  population  is  generated  the  fitness  of  the  individuals  in  the 
population  must  be  determined.  There  are  a  number  of  different  ways  to  confute  fitness, 
but  in  general,  most  of  the  methods  involve  measuring  the  performance  of  a  meniber  of  the 
population  in  relation  to  a  predetermined  set  of  fitness  cases.  As  a  guide,  the  set  of 
fitness  cases  must  be  representative  of  the  entire  domain  since  they  wiU  serve  as  the  baas 
for  achieving  a  generalized  resuh  (Koza,  1992,  pg.  95). 

After  the  fitness  of  the  initial  random  population  is  calculated,  the  genetic 
operations  of  rqrroduction  and  crossover  are  performed.  In  genetic  programming,  the 
rqrroducdon  and  crossover  operations  are  deagned  to  happen  as  separate  evorts  rather 


than  as  pans  of  a  two-stq)  process  as  with  the  genetic  algorithm.  Reproduction  occurs  by 
copying  an  S-erqrression  from  one  generation  to  the  next.  Candidates  for  reproduction 
are  selected  by  u^g  the  fitness-proportionate  n^od  or  a  second  method  known  as 
tournament  selection.  In  tournament  selection,  a  qrecified  nuiid>er  of  population  moidrers 
are  randomly  selected  with  replacement.  The  individual  with  the  best  fitness  is  then 
rq)roduced  in  the  next  generation.  The  sanqrling  with  replacement  is  what  ensures  that 
fitter  individuals  survive  into  succeeding  generations.  (Koza,  1992,  pg.  100) 

The  crossover  operation  in  genetic  programming  is  more  conqrlex  than  that  of 
the  genetic  algorithm  due  to  the  differences  in  the  structure  of  the  population  members. 
The  crossover  candidates  are  selected  using  the  same  method  chosen  for  reproduction. 
This  ensures  that  crossover  occurs  between  individuals  in  a  manner  proportionate  to  their 
fitness.  After  two  individuals  have  been  selected  a  separate  random  number  is  chosen  for 
each  individual  which  corre^onds  to  a  point  in  each  structure.  It  is  at  these  points  that 
the  crossover  operation  takes  place  by  swapping  subtrees.  (Koza,  1992,  pg.  101)  Figure 
2-4  shows  an  exanq))e  of  the  crossover  operation.  The  previously  described  property  of 
closure  on  the  fimction  set  ensures  that  any  re^ihing  S-e?q)ressions  wiU  be  legitimate 
program  representations.  Since  the  points  selected  for  crossover  are  most  likely  to  be  at 
diffkoit  levels  in  each  structure  there  are  a  variety  of  stuations  that  may  resuh.  (Koza, 
1992,  pg.  103)  An  individual  may  reproduce  with  itself  and  produce  two  totally  new 
structures.  If  the  crossover  point  happois  to  be  at  the  root  of  on  S-expression,  that  entire 
S-expression  will  become  a  subtree  on  the  other  parent.  If  the  root  is  selected  in  both 
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Figure  2-5  The  Genetic  Programming  Crossover  Operation 


parents,  they  are  both  just  copied  into  the  next  generation.  The  constraint  on  maximum 
structure  depth  is  the  only  major  limit  on  the  crossover  operation. 

The  third  genetic  operation  of  mutation  is  rarely  used  in  genetic  programming. 
The  normal  argument  for  hs  use,  in  order  to  promote  genetic  diversity,  is  not  as  iiiq>oitant 
v^en  one  considers  the  variety  of  structures  with  different  azes  and  orientations  that  are 
fikely  to  be  obtained  from  the  crossover  process.  Additionally,  ^en  the  crossover  points 
in  both  paroits  correspond  to  endpoints  in  each  structure,  the  effect  is  sim'lar  to  a 
mutation  at  a  single  point.  (Koza,  1992,  pg.  106) 
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3.  Applicatioas 

Due  to  its  relative  youth  as  « leclmique,  applications  using  genetic  programming 
are  currently  confined  to  mostly  experimental  problems.  Koza  does  present  in  his  text  a 
copious  amount  of  exanqiles  of  applying  his  technique  to  a  wide  range  of  problems, 
infthidin£  symbolic  regresson,  game  playing  strategies,  decision  tree  induction,  and 
artificial  life  (Koza,  1992,  pg.  12-14).  In  each  case,  a  domain  ^ecific  function  set  with 
the  proper  closure  was  determined  in  advance.  Once  the  function  set  and  the  appropriate 
fimess  measure  w^s  ^ecified,  all  of  the  problems  proceeded  in  the  identical  manner  using 
a  Hnmam-tndependent  mq)lementation  of  the  mechanical  operations  of  evolution.  An 
intriiil  population  was  generated  at  random  and  the  genetic  operations  of  rq>roduction  and 
crossover  were  used  to  create  succeeding  generations.  Koza  was  able  to  show  via 
eiiq)irical  methods  that  genetic  programming  succeeded  in  each  case  to  find  a  solution  that 
satisfied  the  appropriate  fitness  measure  (Koza,  1992,  pg.  4).  This  ability  to  solve 
problems  over  a  wide  variety  of  domains  by  u^g  a  domain-independent  method  for 
seardiing  the  ^ace  of  possible  conqtuter  programs  to  find  an  individual  computer 
program  is  wiiat  makes  genetic  programming  an  exciting  new  method  of  discovery  (Koza, 
1992.  pg.  8). 
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m,  DESIGN  OF  EXPERIMENTS 


A.  DATA  SELECTION. 

The  COCOMO  data  set  used  for  training  and  testing  the  machine  learning  tedmiques 
examined  in  this  thesis  is  dra>vn  from  pages  496  and  497  of  Boehm's  book.  Software 
Engitieering  Economics.  The  sixty-three  projects  used  to  develop  the  three  versions  of 
COCOMO  are  summarized  in  this  data  base.  This  data  includes  the  project  type,  year 
developed,  the  de'/elopment  language,  the  values  of  the  15  cost  drivers,  the  development 
mode,  and  the  various  COCOMO  estimates.  The  Kemmerer  project  data  was  provided  by 
the  author  in  electronic  form  and  is  included  in  Appendix  C.  The  e>q;>erimental  approach 
taken  for  each  machine-learning  technique  conasted  of  a  two-step  process  that  required  a 
different  training  and  testing  data  set  for  each  step.  The  first  step  used  only  the 
COCOMO  data  partitioned  into  training  and  testing  data  sets.  The  second  step  consisted 
of  using  the  COCOMO  data  for  training  and  the  Kemmero’  data  for  testing.  The 
partitioning  of  the  COCOMO  data  set  in  the  first  step  required  two  tradeoff 
considerations. 

The  first  tradeoff  involved  the  ^  of  the  training  and  testing  data  sets.  Machine 
learning  methods  perform  at  their  best  when  they  have  a  large  amount  of  data  available  for 
training  that  is  represoitative  of  the  population,  or  problem  domain.  In  fiict  a  good 
guideline  is  more  is  always  better.  However,  there  are  only  63  projects  available  in  the 
COCOMO  data  base  in  the  text.  In  this  case,  choosing  to  use  too  much  data  for  training 
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reduces  the  nuiid)er  of  data  sets  available  for  testing  and  trivializes  the  resuhs  beyond  a 
certain  point.  Ahematively,  choosing  too  little  data  for  training  reduces  the  effectiveness 
of  the  machine  learning  techniques.  A  balance  was  struck  by  deciding  to  break  the  data 
into  thirds,  with  two*thirds  to  be  used  for  training  and  one>third  to  be  used  for  testing. 
This  was  acconq)lished  by  transcribing  the  data  set  from  the  text  into  a  Quattro  Pro 
^read^eet  and  then  sorting  the  projects  by  size  and  type.  Once  the  data  was  sotted  in 
this  manner,  a  random  number  between  one  and  three  was  assigned  to  each  project.  The 
data  was  then  partitioned  into  groups  based  on  the  random  number  assigned  and  each 
group  was  analyzed.  This  cursory  analysis  showed  that  for  each  group  the  mean  project 
size  in  thousands  of  delivered  source  instructions  (KDSI),  the  average  year  of 
development,  and  the  development  mode  proportions  were  roughly  similar,  indicating  a 
reasonable  distribution.  After  this  analysis,  42  of  the  projects  were  identified  as  the 
training  set  and  21  projects  were  identified  as  the  testing  set.  These  data  sets  are  included 
in  Appendix  D.  The  same  training  and  testing  sets  were  then  used  for  all  of  the  machine 
learning  tediniques  examined.  The  conqroshion  of  the  training  and  testing  data  sets  for 
each  set  of  experiments  was  also  kept  as  identical  as  possible  and  consisted  of  the  project 
size,  aimual  adjustment  factor,  development  mode,  and  the  fifteen  COCOMO  cost  drivers, 
with  limited  exceptions. 

One  effect  of  partitioning  the  data  in  this  manner  is  that  the  testing  data  set  may  not 
be  complete^  diaracteristic  of  the  "population”.  Some  testing  set  projects  contained 
some  cost  driver  values  that  were  not  contained  in  the  training  data  set.  This  situation 
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describes  the  second  tndeoff  Hand  selecting  the  data  to  ensure  that  every  cost  driver 
value  represented  in  the  data  base  was  in  the  training  set  was  initially  considered  but 
rejected  for  two  reasons.  First,  it  would  probably  be  meaningless  ^^en  considering  that 
there  are,  at  a  nwnimnin,  four  values  for  each  cost  driver.  For  fifteoi  cost  drivers  this 
means  there  are  at  least  4'^  possible  combinations  of  cost  drivers,  and  there  are  only  63 
projects  in  our  data  set.  Secondly,  on  a  more  practical  basis,  several  of  the  cost  driver 
values  are  only  represented  once  in  the  project  data  base,  so  if  they  were  included  in  the 
training  data  there  would  be  no  way  of  examining  the  effectiveness  of  this  measure. 
Therefore,  there  appeared  to  be  no  benefit  in  ensuring  that  every  cost  driver  value  was 
fiilly  represented  in  the  training  set. 

B.  NEURAL  NETWORK  PROCEDURES 

1.  Neural  Network  Software 

The  software  used  to  develop  the  neural  networks  in  this  thesis  is  BrainMaker 
Ptofessional  v2.5,  which  is  designed  and  marketed  by  California  Scientific  Software,  Inc. 
This  software  is  a  DOS-based  product  manufoctured  for  use  on  IBM-conqratible  personal 
con^ruters.  BrainMaker  is  designed  to  construct,  train,  test,  and  run  backpropagation 
neural  networks.  It  is  a  moiu-driven  program  that  provides  the  user  a  large  range  of 
control  over  the  design  and  operation  of  a  neural  network.  BrainMaker  comes  with  an 
extensive  manual  as  well  as  a  corr^anion  program  called  NetMaker  which  can  be  used  to 
prq)are  the  data  files  needed  by  BrainMaker.  All  of  the  files  used  by  BrainMaker  are  in 
standard  ASCII  format,  so  thQ^  can  be  prq^ared  using  most  any  text  editor  if  the  user  so 
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desires,  although  usmg  the  NetMaker  program  ^eeds  up  the  process  since  it  heh)s 
automate  the  generation  of  BrainMakei's  inputs. 

BrainMaker  requires  two  files  at  a  minimum  to  train.  The  first  file  is  the 
network  definition  file,  which  tells  BrainMaker  the  key  network  parameters,  sudi  as  the 
number  of  input  and  output  neurons,  the  number  of  hidden  layers  and  hidden  neurons,  the 
input  and  output  data  definitions  and  ranges,  the  learning  rate,  and  the  type  of  transfer 
fimction  the  nonons  will  use.  An  example  of  a  network  definition  file  is  shown  in  Figure 
3-1.  The  second  file  that  BrainMaker  requires  is  the  fact  file.  This  is  the  file  that  contains 
the  data  the  network  will  use  for  training.  This  file  consists  of  ahemating  rows  of  data, 
with  the  first  row  representing  one  input  set  and  the  second  row  representing  one  output 
set.  An  example  of  a  fiict  file  is  contained  in  Figure  3-2.  Additional  input  files  that  may  be 
included  are  a  testing  fact  file  and  a  nmning  &ct  file.  The  testing  fiia  file  con^s  of 
ahemating  rows  of  input  and  output  data  not  included  m  the  training  fiict  file.  This  data 
may  be  used  during  the  training  process  to  judge  the  proqiective  performance  of  the 
network  at  user-defined  intervals  during  the  training  process.  When  a  testing  fact  file  is 
q>ecified,  BrainMaker  pauses  at  the  ^ecified  interval  and  tests  the  current  network 
configuration  u^g  the  fiicts  in  the  testing  fiict  file.  The  resuhs  of  these  tests  along  with 
some  associated  error  statistics  are  written  to  an  output  file.  The  training  statistics  for 
eadi  run  may  also  be  wiittoi  to  a  separate  file  if  this  option  is  activated.  The  running  fiict 
file  provides  the  user  with  the  capability  to  test  the  trained  network  with  new  sets  of  inputs 
in  a  batdi  manner  and  write  the  resuhs  to  an  output  file  in  a  user-specified  format. 
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2.  Neural  Network  Design 


The  neural  network  design  and  training  process  is  very  iterative.  Since  any 


netw^rii  configuratitm  wfll  learn  some  training  &cts  to  the  user-specified  level  of  tolonnce 


during  the  training  process  it  is  irrqrortant  to  have  a  strategy  for  determining  a  broadly 


effective  configuration  when  beginning  the  network  design  process. 


input  nunAer  1  20 
output  number  1 1 
hidden  16  19 

filename  train&cts  c;\brain\testVcltrain.fct 

filename  testfacts  ci^brainNtestVcltnun.tst 

leamrate  0.9000  50  0.75  75  0.6000  90  0.5000 

leamlayer  1.0000  1.0000  1.0000 

traintol  0.1000  0.04  0.8000  100 

testtdl  0.2000 

random  5.0 

maxruns500 

testruns  1 

fiinction  hidden  1  sigmoid  0.0000  1.0000  0.0000  1.00000 

fiinction  hidden2  sigmoid  0.0000  1.0000  0.00001.00000 

fiinction  output  sigmoid  0.0000  1.0000  0.00001.00000 

dictionaiy  input  IXDG  KDSI AAF  RELY  DATA  CPLX  TIME  STOR  VIRT  TURN 

ACAP 

AEXPPCAP  VEXPLEXPMODPTOOL  SCED  E  ORG  SD 
dictionary  output  LOG_MMAC 
scale  input  minimum 

0.47712  0.43000  0.75000  0.94000  0.70000  1.00000  1.00000  0.87000  0.87000 
0.71000  0.82000  0.70000  0.90000  0.95000  0.82000  0.83000  1.00000  0.00000 
0.00000  0.00000 
scale  input  maximum 

2.66651  1.00000  1.40000  1.16000  1.30000  1.66000  1.56000  1.30000  1.15000 
1.19000  1.29000  1.42000  1.21000  1.14000  1.24000  1.24000  1.23000  1.00000 
1.000001.00000 
scale  output  minimum 
0.77815 


F^ore  3-1:  A  sample  network  definition  file 


Ihere  is  no  way  to  know  which  network  configuration  will  be  the  most 


successful  in  learning  the  data  prior  to  the  training  process.  At  best,  some  authors 
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recoimiiend  genial  guideUnes  for  network  deagn  that  can  serve  as  a  starting  point.  The 
makers  of  BrainMaker  suggest  first  that  there  is  no  little  reason  to  have  a  network  with 
more  than  two  hidden  layers  (BrainMaker,  1992,  pg.  14*4).  They  state  in  thefr  manual 
that  they  have  never  observed  a  network  with  more  than  two  hidden  layers  that  could  not 
also  be  trained  cm  just  two  byers.  As  for  the  number  of  neurons  in  the  hidden  layers,  there 
are  two  suggestions.  The  makas  of  BrainMaker  (1992)  suggest  using  the  sum  of  the 
hqrut  and  output  neurons  divided  by  two,  vdule  the  authors  Eberhart  and  Dobbins  (1990) 
suggest  using  the  square  root  of  the  sum  of  the  input  and  output  neurons  phis  a  coiqile. 
Both  of  these  guidelines  are  based  on  enqrirical  observations,  not  any  underlying  principle. 


In  the  end,  it  is  necessary  to  experiment  with  a  variety  of  networks  to  determine  a 


successful  configuration.  This  is  probably  the  most  time*consuming  portion  of  the  design 


process  but  BrainMaker  has  a  method  for  automating  this  process  to  a  certain  extent. 


&cts 
- 1 

1.62324  0.96  1.4  1.08  1.3  1.48  1.56  1.15  0.94  0.86 

0.82  0.86  0.9  1  0.91  0.91  1  (  E 

2.78175 

1.36172  0.96  1.15  1.08  1  1.06  1  1  0.87  1  1 

1  1  1  0.91  1.1  1.23  (  E 

2.36172 

1.79239  0.81  1.4  1.08  1  1.48  1.56  1.15  1.07  0.86  0.82 

0.86  1.1  1.07  1  1  1  [  E 

3.02653 

1.57978  1  1.15  1.16  1.3  1.15  1.06  1  0.87  0.86  1 

0.86  1.1  1  0.82  0.91  1.08  [  E 

2.7185 


Figure  3-2  A  Sample  BrainMaker  Fact  File 
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3.  Pk’cparatiooi  for  Training 


A  key  a^ect  m  prq)aring  to  train  a  networic  is  data  preparation.  Due  to  the 
nature  of  neural  networks  and  the  way  that  the  training  tolerance  is  calculated,  it  is 
necessary  to  pay  particular  attention  to  the  way  the  data  is  presented  to  the  networiL 
First,  all  iiq)ut  variables  diould  vary  over  a  roughly  range  betwea  the  rnmimuin 
and  nuximum  values.  The  reason  for  this  is  fiurly  straightforward.  Since  eveiy  iiq>ut  is 
connected  to  each  neuron  in  the  first  hidden  layer,  a  mgle  input  that  varies  ovo’  a  large 
range  can  tend  to  "drown  out"  the  other  inputs,  diminishing  their  contribution.  Secondly, 
since  the  iqiuts  and  outputs  to  the  network  are  all  normalized,  data  that  varies  over  a  wide 
range  reduces  the  ability  of  the  network  to  distinguish  small  changes.  For  exait^le,  if  a 
significant  portion  of  a  particular  piece  of  data  ranges  between  500  and  600  while  a  finv 
values  range  between  10  and  20,  the  network  wQl  have  difficulty  determining  the 
difference  betweoi  10  and  20.  As  a  rule  of  thumb,  smaller  dianges  are  better.  This 
becomes  particularly  inq)ortant  when  understanding  the  concq)t  of  network  training 
tolerance. 

Tolerance  is  the  statistic  used  to  determine  the  level  of  precision  to  which  the 
network  trains.  Tolerance  is  e^qiressed  as  a  percentage  of  the  differoice  between  the 
minimiiin  and  maytmiim  output  vahies.  As  an  exan:q>le,  if  the  minimum  output  is  10  and 
the  maximum  output  is  1000,  then  a  tolerance  of  0.1  means  that  any  network  output  that 
611s  within  0.1*(1000-100) »  90  of  the  actual  output  value  will  be  congdered  correct  by 
the  network  Therefore,  if  the  output  training  vahies  vary  over  a  large  range,  the  accuracy 
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of  the  oetworii  w31  be  diiniiiuhed.  This  required  two  transfomutions  to  the  COCOMO 
date  for  use  with  the  neural  networiL  Since  the  KDSl  values  and  the  actual  inan*ni(niths 
vary  over  a  large  range,  logarithins  of  these  two  values  were  used,  ^ch  greatly  reduced 
the  difference  between  the  mininiuin  and  maxiniuin  values  and  increased  the  ability  of  the 

network  to  kam  and  predict  project  ^ort. 

4.  Automating  the  Training  Process 

BrainMaker  has  an  option  whidi  allows  the  user  to  specify  a  range  of  network 

hidden  layer  configurations,  as  well  as  other  parameters,  that  can  be  tested  in  a  variation  of 
a  brute  force  attack  (BrainMaker  GTO,  1993,  pg.  5).  This  option  was  used  to  test  three 
ranges  of  network  configurations  that  were  used  in  an  iterative  process  to  search  for  the 
best'performing  network  configurations.  The  three  ranges  tested  are  suimnarized  in  Table 
3-1: 


Table  3>]  Network  Ranges  Tested 


Number  of  Hidden  Neurons 

Number  of  Lajrers 

ItolO 

1  and  2  hidden  layers 

10  to  20 

1  and  2  hidden  leyers 

IS  to  23 

1  and  2  hidden  layers 

Ihe  use  of  this  option  requires  the  user  to  o-eate  an  additional  input  file  b^^d 
die  usual  training,  testing,  and  definition  files.  This  setiq>  file  contains  all  the  network 
configuration  parameters  and  the  ranges  of  those  that  are  bmg  varied,  b  this  study,  the 
oofy  parameters  varied  were  the  number  of  neurons  and  hidden  layers,  mce  these  two 
fiictors  have  the  greatest  effect  on  overall  network  performance.  The  remaining  network 
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parameters  (v^ch  deal  mostly  whh  the  learning  rate  associated  with  the  Delta  rule)  were 
fixed  with  the  excq>tion  of  training  tolerance,  wfaidi  was  allowed  to  decrease  in  a  stq)wise 
pattern  from  0. 1  to  a  limit  of  0.04.  The  stepwise  decreases  in  tolerance  are  triggered 
udiea  the  networit  learns  all  the  training  frets  at  the  current  training  tolerance.  The  testing 
tolerance  was  q>ecified  as  0.2.  An  example  of  the  setiq>  file  for  this  option  is  diown  in 
Figure  3-3: 

The  training  and  testing  data  files  consisted  of  the  COCOM O  data  modified  as 
mentioned  previously  by  changing  the  KDSI  and  effort  values  into  logarithmic  vahies. 
There  were  42  training  patterns  and  21  testing  patterns.  The  network  definhkm  file 
ctmsisted  of  a  generic  network  definhion  file  that  provided  the  ranges  of  the  input  values 
and  the  variable  names,  similar  to  the  file  shown  in  Figure  3-1. 

Once  the  three  ranges  of  configurations  had  been  tested  and  aO  of  the  resuhs 
written  to  three  output  files,  the  top  performing  networks  were  selected.  The  selections 
were  made  by  reading  all  of  the  output  files  into  Quattro  n’o  for  Windows  ^readsbeets 
and  sorting  the  resuhs  based  on  the  number  of  correct  outputs  for  both  the  training  and 
testing  data  sets.  After  the  sorting  process,  the  networks  drown  m  Table  3-2  were  diosen 
fen  further  training  at  a  ntore  detailed  level  The  choice  of  this  iterative  process  was  based 
upon  several  fretors  that  mainly  involved  hardware  and  software  constraints.  The 
hardware  constraint  concerned  the  disk  qrace  required  to  save  the  testing  results,  hr  the 
initial  round  of  testing,  the  statistics  for  the  testing  dau  set  were  written  to  didc  every  25 
rims.  Eadi  of  these  runs  took  approximately  four  to  five  hours  on  a  33  MHz  486 
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filenames  c;\biain\thetis\cltnin.def  c;\brain^besis^ltraiB.fct 

c:\bnin\diesis\cltraiattt  GTO.STA  c.^nin\thetu\tcst4.  ACC 

inaxnuis400 

tettnins25 

sepmie  0.2000 

cunent  111 

■tamoi  0.10000  0.10000  o.ooooo  o.ioooo 

endid  0.04000  0.04000  0.00000  0.04000 

tolmuh  O.tOOOO  O.SOOOO  0.00000  O.tOOOO 

toipcs  1001000100 

hiddenl  1 10  1 10 

hid(kii2  1  10  1 10 

hiddenla^ere  3  1 

decfcaae  0.100000.100000.00000  0.10000 
ad^uns  0000 

inputmin  0.00000  0.00000  0.00000  0.00000 
ftinctionmin  0.000000.00000  0.00000  0.00000 
gain  1.000001.00000  0.000001.00000 
noise  0.000000.000000.000000.00000 
blurring  1  0 

random  S.OOOOO  S.OOOOO  0X0000  3.00000 
delay  1  0 

leamrate  1.000001.000000.000001.00000 
Icaminit  0.90000  0.900000.000000.90000 
leampctl  3050050 

leamrate]  0.73000  0.73000  0.00000  0.73000 
leampct2  73  73  0  73 

Ieaniraic2  0.600000.600000.000000.60000 
Icarapct3  9090  0  90 

leamratea  0.300000.30000  0.00000  0.30000 
leamlayerl  1.000001.000000.000001.00000 
leanilayer2  1.000001.000000.000001.00000 
leamliO«rout  1.000001.000000.000001.00000 
smooching  0.90000  0.90000  0.00000  0.90000 
smoothlt^l  0.900000.900000.000000.90000 
smooUilayerl  0.90000  0.90000  0.00000  0.90000 
smoothlayerout  0.90000  0.90000  0.00000  0.90000 


Figure  3-3  A  Mmple  network  conflguratioD  testing  file 
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TaUc  3-2  Top  Network  Coofiguratioiii 


First  Hiddm  Layer  Ncuroas 

Sccead  Hiddca  Layer  Neareas 

5 

8 

t 

9 

10 

11 

16 

19 

IS 

IS 

24 

25 

con^uter  and  took  approximately  300  kilobytes  of  disk  q>ace.  Saving  the  resuks  for 
every  run  would  have  risked  filling  aO  the  disk  space  available  on  the  disk  and  risked 
cradling  the  computer  while  unattended,  wkich  would  have  wasted  a  training  tun. 
Additionally,  the  resuking  data  files  would  have  been  too  large  to  load  into  a  spreadsheet 
in  order  to  perform  the  sorting  operations.  This  is  the  reason  the  training  was  broken  into 
two  cycles,  with  the  goal  of  the  first  cycle  being  to  identify  the  networks  with  the  best 
performance  at  a  nucro  level  The  criteria  that  was  used  to  detemnne  wdiich  networks  to 
choose  for  further  training  was  based  on  which  networks  correctly  predicted  aD  21 
projects  in  the  testing  data  set  at  some  point  during  their  training  period.  Six  network 
configurations  met  this  okoia.  Once  these  six  best  network  configurations  were 
identified,  the  second  round  of  training  was  performed.  For  this  training  cycle,  network 
definition  files  were  mated  for  eadi  of  the  six  configurations.  AO  parameters  in  these 
definition  files  wme  identical  excqit  for  the  number  of  neurons  in  the  two  hidden  layers. 
For  these  networks,  the  c  ^hnum  number  of  training  runs  was  set  to  500.  A  tun  is  this 


44 


tense  is  defined  as  t  pass  through  the  entire  training  data  set  in  which  fitness  statistics  are 
accumulated  for  the  cunent  configuration  and  weight  vector  set.  The  COCOMO  training 
and  testing  files  were  provided  as  inputs.  Training  and  testing  statistics  for  all  500  runs 
tvere  accumulated  in  sq>arate  files  for  each  of  the  six  configuratimis.  Once  this  cyde  of 
tnining  was  over,  the  testing  and  training  statistics  files  were  again  read  into  Quattro  Pro 
for  Windows  q>readsheets  and  analyzed  to  determine  the  optimum  nuinber  of  training  runs 
for  eadi  of  the  six  configurations.  This  optimum  point  was  based  on  the  run  where  all  42 
projects  in  the  training  data  were  correctly  predicted  and  the  network  training  toloance 
was  at  a  minimum  for  the  training  cycle.  When  this  point  was  determmed  for  each 
netwoih,  the  training  process  was  repeated  fi'om  the  beginning  up  to  thi«  optimum 
performance  point.  The  analysis  of  neural  network  performance  is  based  iq}on  the  resuhs 
obtained  fi'om  these  final  versions  of  the  six  networks.  The  definition  files  for  these 
networks  are  shown  in  Appendix  E. 

This  entire  netwoih  training  process  starting  with  the  six  network  configuratitms 
was  treated  a  second  time  in  order  to  test  the  performance  of  the  neural  network  on  a 
data  set  separate  fi-om  Boehm's  COCOMO  data  set.  In  this  second  round  of 
experimentation,  aU  of  the  63  COCOMO  projects  were  used  as  the  training  set,  and  the  15 
projeas  that  make  op  the  Kemmerer  data  set  were  used  as  the  testing  set. 
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C  GENETIC  ALGORITHM  PROCEDURES 

1.  Gcoetic  Aigoritbiii  Goal 

The  goal  of  the  second  group  of  experimeDts  in  this  thesis  was  to  use  a  ge&edc 
algorithm  to  deteimme  an  optimum  set  of  values  for  the  cost  drivers,  coeflScients,  and 
exponents  for  the  Intermediate  COCOMO  model  In  this  grotq>  of  experiments  the  mputs 
to  the  genetic  algorithm  mchided  the  Intermediate  COCOMO  model  structure, 

MM  «  Effort  Adjustment  Factor*COEFF*(Adjuited  ICDSI)**EXP 

and  the  partitioned  COCOMO  data  with  two  modifications.  The  first  modificatitm  to  the 
COCOMO  training  and  testing  data  set  prior  to  usmg  the  genetic  algorithm  consisted  of 
muh^lying  the  aimual  adjustment  fiictor  and  KDSI  values  together  to  obtain  the  adjusted 
KDSL  This  action  was  takes  ance  the  Annual  Adjustment  Faaor  provided  no  addirional 
information  to  the  genetic  algorithm  process  in  the  absence  of  information  on  how  it  was 
derived.  The  second  modification  consisted  of  expressing  the  cost  drivers  by  their  literal 
values,  sudi  as  HIGH,  LOW,  or  NOMINAL,  vice  their  numeric  values. 

2.  Genetic  Algorithm  Software 

The  gmetic  algorithm  software  package  used  was  GAucsd  1.4,  a  Olanguage 
genetic  algorithm  developed  by  bficol  Sduraudo^b  of  the  University  of  Cahfomia,  San 
Diego,  based  on  previous  woric  by  John  Grefoistette  of  the  Naval  Researdi  Laboratory. 
Die  uncongiiled  C  code  for  GAucsd  1.4  is  publicly  available  via  anonymous  on  the 
fiMemet  and  can  be  adapted  to  any  madiine  that  has  a  C  corqiQer.  hi  this  case  the  target 
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plitfonn  was  a  Sun  woricstatioii.  The  choice  of  GAucsd  1 .4  was  based  on  its  availability 
and  ease  of  use.  This  package  is  weU-designed  in  that  the  user  has  only  to  suppl>  a 
dtromosome  structure  and  a  fitness  function  (written  m  C)  that  defines  the  problem. 
GAucsd  1.4  provides  the  rest  of  the  fimctionahty  required  in  a  genetic  algorhhm,  such  as 
population  initialization,  crossover,  and  mutation.  GAucsd  1.4  also  provides  an  awk 
language  macro  that  integrates  the  user  defined  fimess  function  and  adds  in  the  code 
required  to  implement  the  genetic  algorithm  operations.  This  design  feature  relieves  the 

user  Df  the  task  of  code  integration.  (Schraudo^^h  and  Grefienstette,  1992,  pp.  9-12) 

3.  GAucsd  Preparation 

The  duomosome  structure  defined  in  the  genetic  algorithm  fimess  function  used 
in  this  study  consisted  of  a  string  of  96  numbers  that  represented  all  of  the  cost  driver 
values,  along  with  the  coefiBcients  and  e}q)onents  for  all  three  project  developimst  modes. 
Since  there  are  a  maximum  of  six  values  for  eadi  cost  driver  and  a  total  of  fifteen  cost 
drivers,  the  first  90  numbers  represented  the  cost  drivers.  Although  most  cost  drivers 
have  less  than  six  values,  a  total  of  six  values  were  reserved  for  eadi  driver.  This  aided  in 
the  structure  of  the  problem  by  allowing  the  indexing  process  to  operate  as  if  the  first  90 
numbers  represented  a  pseudo  6x15  array.  The  excess  cost  driver  values  in  no  way 
impeded  the  fimctionality  of  the  genetic  algorithm.  The  numeric  values  derived  by  the 
genetic  algorithm  were  substituted  for  the  literal  values  in  the  COCOMO  training  set 
v^ch  were  stored  as  index  values  in  a  C  table  structure.  These  first  90  values  were 
allowed  by  the  fimess  functitm  to  range  from  0.5  to  2.0.  The  91st  through  93rd  numbers 
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rqireseDted  the  coefiScieots  for  e&wh  developmeot  mode  and  the  94th  through  96th 
numbears  rq>reseoted  the  e}q)ODents  for  eadi  development  mode.  These  last  ax  values 
were  allowed  by  the  fimess  function  to  range  from  0.0  to  4.0.  The  dioice  of  these  ranges 
was  somewdiat  arbitrary  but  allowed  for  a  greater  range  of  values  than  those  expres.ed  in 
Bodun's  Intermediate  COCOMO  modeL  The  actual  fimess  measure  for  eadi  iDend>er  of 
the  population  was  initially  the  average  relative  error  of  the  actual  effort  in  the  COCOMO 
training  data  set  versus  the  estimated  effort  calculated  by  the  fimess  function.  This 
estimated  effort  was  calculated  using  the  Intermediate  COCOMO  model  as  a  tenqrlate. 

The  genetic  algorithm  derived  numeric  values  for  the  hteral  cost  driver  values,  along  with 
the  coefficient  and  erqronent  values  appropriate  for  each  project's  development  noode  then 
were  used  as  inputs  into  the  Intermediate  COCOMO  model.  The  capability’  to  impose 
constraints  on  the  values  generated  by  the  genetic  algorithm  was  also  provided  and  is 

expanded  upon  later.  The  GAucsd  user«defined  fimess  fimction  is  shown  in  Appendix  F 
4.  Initial  Genetic  Algorithm  Benchmarking 

Along  with  the  user*defined  fimess  function,  the  other  input  to  the  GAucsd  1.4 

program  is  the  defauh  file  that  specifies  population  size,  crossover  rate,  mutation  rate,  and 
number  of  generations.  The  first  step  in  umg  the  genetic  algorithm  centered  on  finding  an 
effective  population  size  and  crossover  rate  to  use  with  the  960  bit  chromosome.  This 
was  accomplished  by  a  series  of  test  nms  umg  various  population  sizes  and  otrssover 
rates  while  keeping  mutation  rate  and  number  of  generations  constant.  The  test  tuns  are 
summarized  in  Table  3-3. 


48 


Table  3-3  Genetic  Algorithm  Benchmarii  Tests 


Fopulatioo  Sizes  Tested 

too.  200, 1000 

Crossover  Rates  Tested 

0.5.1.5.2.5,30.5.0 

MutatiooRate 

0.00002 

Number  of  Generatioiu 

2.000 

The  training  data  for  this  benchmarking  process  consisted  of  the  entire 
COCOMO  projea  data  set.  The  entire  set  was  used  to  examine  how  weD  the  values 
derived  from  the  genetic  algorithm  performed  reject  to  the  values  determined  by 
Bodim  in  his  original  model.  The  resuhs  of  this  initial  round  of  benchmarking  drowed 
that  a  population  size  of 200  and  a  crossover  rate  of  either  3.0  or  5.0  performed  the  best 
in  ahoost  every  case.  The  crossover  rate  greater  than  one  meant  that  the  chromosome 
underwent  several  crossovers  in  each  generation.  A  sanqrle  of  the  genetic  algorithm 
defruk  file  is  drown  in  Figure  3-4. 

After  the  optimum  population  dze  and  crossover  rates  were  determined,  nine 
versions  of  the  fitness  fimetion  were  prqrared.  These  versions  differed  based  on  how  the 
fitness  of  eadi  meniber  of  the  population  was  measured  and  how  the  previously  mentioned 
constraints  were  applied.  The  three  fimess  measures  used  in  the  erqreriments  were 
average  relative  error,  mean  error,  and  mean-squared  error  and  these  measures  woe 
calculated  as  shown  in  Table  3-4. 
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Figure  3-4  A  Sample  Genetic  Algorithm  Default  File 

Muh^le  measures  of  fitness  were  tested  due  to  the  general  nature  of  adaptive 
algorithms,  of  wUch  the  genetic  algorithm  is  an  excellent  exaiig)le.  Different  measures  of 
fitness  cause  different  bdiaviors  in  the  algorithms  and  affect  the  learning  process. 
Mean-squared  error,  for  instance,  eiq)haazes  large  magnitude  errors  at  the  expense  of 


Table  3-4  Genetic  Algorithm  Fitness  Measures 


Fitness  Mcamre 


Average  Relative  Enor 


Mean  Error 


Mean-squared  Error 


Formula 


ZAbs((Estimal*d-Actuaf)/Actua. 


Number  of  fitness  cases 


^AbsiEstimattd-Actuai 


Number  of  fitness  cases 


HEsttmattd- Actual)* 


Number  of  fitness  cases 


SO 


mailer  magnitude  errors.  This  type  of  behavior  csused  the  genetic  algorithm  to  focus  on 
learning  the  larger  software  projects  that  had  large  errors  at  the  expense  of  the  smaller 
pngects.  Hiis  bdiavior  led  to  poor  resuhs.  Mean  error  reduced  this  tendency  to  a  degree, 
but  the  best  learning  performance  was  found  when  usmg  average  relative  error.  In  these 
mqreiiments,  this  fimess  measure  equalized  the  effort  the  genetic  algorithm  erqrended 
katning  all  of  the  projects  smce  error  was  expressed  as  the  average  percentage  difference 
between  the  estimated  value  and  the  actual  value.  A  secondary  observed  benefit  of  this 
measure  was  that  both  mean  error  and  mean*squared  error  were  also  minimized  when 
using  this  fimess  measure  even  though  neither  of  these  fimess  measures  were  explicitly 

expressed  in  the  fimess  function. 

5.  Genetic  Algorithm  Constraints 

The  final  aspect  of  the  various  fimess  functions  involved  the  imposition  of 
various  degrees  of  constraints  on  the  values  generated  by  the  genetic  algotithia  An 
infection  of  Boehm's  COCOMO  cost  driver,  coefficient,  and  erqronen  values  reveals 
certain  patterns  or  constraints.  For  example,  in  the  case  of  the  COCOMO  erqronents,  the 
magnitude  of  the  exponent  increases  as  the  mode  shifts  from  organic  to  semidetadied  to 
embedded.  Conversely,  the  coefficient  decreases  as  the  same  mode  dufts  occur.  Similar 
patterns  can  be  seen  in  the  values  of  the  cost  drivers.  When  injecting  the  cost  driver 
table,  the  magnitude  of  the  first  seven  cost  drivo’s  increases  wfren  moving  from  a  value  of 
Very  Low  to  a  value  of  Extra  Ifigh.  The  opposite  movemoit  occurs  in  drivers  d^t 
through  fourteen,  fri  this  case  the  magnitude  of  the  driver  decreases  as  the  value  shifts 
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from  Veiy  Low  to  Very  Ifigh.  The  vahie  of  the  finil  cost  driver,  required  development 
sdiedule  (SCEDX  moves  in  a  high*low-high  manner  as  the  value  diifts  from  Very  Low  to 
Very  Hi^.  These  patterns  can  be  interprrted  as  constraints  that  the  cost  driver  values 
friould  ob^.  To  test  v^hether  or  not  these  constraints  added  to  the  ability  of  the  genetic 
algorithm  to  learn  the  project  data  and  find  an  optimal  set  of  values,  these  constraints  were 
end)edded  in  three  versions  of  the  fitness  function.  In  the  first  veraon,  no  ctmstraints  were 
placed  on  the  order  of  the  coA  drivers,  exponents,  or  coefficients.  In  the  second  version, 
penalty  measures  were  introduced  on  the  coa  drivers  so  that  if  the  previously  identified 
patterns  in  the  fira  14  coa  drivers  (SCED  was  allowed  to  vary  without  constraint)  did  not 
appear  in  the  chromosome,  the  fimess  of  the  individual  was  penalized  based  tq>on  the 
degree  of  non-conq)liance.  In  the  final  version  of  the  constrained  fimess  fimction,  penalty 
measures  were  introduced  on  the  generation  of  the  exponents  and  coefficients  as  wdl  as 
the  coa  drivers. 

6.  The  Genetic  Algorithm  Testing  Process 

The  nine  fimess  functions  used  for  training  the  genetic  algorithm  were 

ctmstruaed  usmg  the  three  mt.aires  of  fimess  and  the  three  constraint  levels  previously 
described.  Testing  was  performed  umg  a  population  aze  of 200  and  crossover  rates  of 
3.0  and  5.0  with  the  nuniber  of  generations  set  at  5000  for  all  teas.  There  were  18  total 
teas  and  each  tea  took  approximately  two  to  four  hours  on  a  Sun  workstation.  The  top 
twenty  chromosomes  in  each  tea  were  saved  to  an  output  file  and  read  into  a  Quattro 
fi>r  Windows  qrreaddieet.  The  chromosome  values  with  the  bea  fimess  in  each  run  were 
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translated  into  their  associated  COCOMO  model  values  using  a  previous  constructed 
aeries  of  linked  q>readdieets.  Hiis  table  of  translated  COCOMO  modd  values  was  then 
linked  to  the  test  data  set  in  such  a  manner  as  to  allow  for  automatic  dating  of  the  test 
data  and  its  associated  performance  measures  whenever  new  chromosome  data  was  pasted 
aito  the  ipreadsheet.  The  results  of  all  18  tests  were  processed  in  the  ««">«  manner. 

A  slightly  different  procedure  to  that  previously  described  was  used  vriien  the 
genetic  algorithm  was  trained  using  all  63  COCOMO  projects  and  tested  with  the 
Kemmerer  project  data.  In  this  case,  the  best  parameter  set  from  the  original  genetic 
algorithm  benchmarking  tests  was  used  along  with  the  full  range  of  constraints  in  the 
fitness  fiinction.  The  use  of  all  of  the  constraints  was  based  upon  the  results  observed  in 
the  first  phase  of  testing.  The  number  of  generations  was  increased  from  the 
bendunarldng  levd  of 2000  to  5000.  The  top  twenty  diromosomes  frt>m  thic  nm  were 
again  saved  to  an  output  file  that  was  loaded  into  a  spreadsheet  holding  the  Kemmerer 
data.  The  performance  of  the  gaetic  algorithm  with  req)ect  to  the  Kemmerer  project  set 
was  then  calculated. 

D.  GENETIC  PROGRAMMING  PROCEDURES 

1.  Genetic  Programming  Software 

The  goal  of  the  final  group  of  eTperiments  in  this  theris  was  to  test  the  ability  of 
genetic  programming  to  derive  an  e)qilicit  cost  estimation  model  given  only  the  COCOMO 
data  and  a  fitness  fimcdon.  The  genetic  programming  software  used  was  SGPC:  Simple 
Genetic  Programming  in  C  by  Walter  Alden  Tackett  and  Aviram  Canm.  This  software 
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is  based  on  the  original  USP  code  published  by  Koza  in  his  book  and  is  available  via 
anonymous  ^  from  the  Santa  Fe  Institute  and  the  University  of  Texas.  This  software 
provides  all  the  fimcdonality  required  to  apply  genetic  programming  to  a  variety  of 
problem  domains.  The  genetic  operations  of  population  initialization,  crossover, 
reproduction,  and  mutation  are  fully  supported  by  this  software.  The  usct  is  only 
re^oosible  for  defining  functions  q)ecific  to  the  problem  at  hand,  ensuring  dosure  of  the 
fimctions,  and  providing  the  appropriate  fitness  function  and  terminal  set.  This  software 
also  has  the  capability  to  train  and  test  simuhaneously,  which  reduces  the 
post-expeiimental  processbg  of  the  output  expressions.  Due  to  the  memory  requirements 
and  CPU  intensive  luture  of  this  application,  the  target  platform  for  this  application  was  a 
Sun  workstation. 

2.  Genetic  Programming  Preparation 

Prqraration  for  the  genetic  programming  process  usng  the  SGPC  software 

hivolved  five  stqrs.  In  the  first  step  the  functions  available  for  use  m  the  S^ergrressions 
were  defined.  Since  this  set  of  expeiimats  was  basically  a  problem  of  regression,  the 
standard  mathematical  operations  of  addition,  subtraction,  muhiplication,  protected 
division,  and  erqronentiation  were  defined  as  (qrerators  in  SGPC.  The  protected  division 
operator  is  a  qtedally  defined  division  operator  that  provides  dosure  for  the  case  of 
division  by  zero.  When  this  event  occurs,  the  operator  is  defined  to  return  a  value  of  one. 

In  the  second  stq>  the  C-language  program  structure  that  provides  the  terminal 
set  (variable  dedarations)  and  the  training  data  was  constructed.  Hus  procedure  was 
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pei£)nDed  usmg  a  sa^t  written  in  peri  that  took  the  tab-dehmited  ASCII  file  containing 
die  COCOMO  training  data  and  buih  a  C  source  code  file.  This  C  source  code  file  made 
the  variable  dedaratitHis  that  defined  the  temsnal  set  and  created  the  table  structure 
containing  die  training  data.  The  training  data  for  this  set  of  experiments  ctmasted  of  the 
tame  projects  as  those  used  for  the  previous  experiments.  The  first  round  of  erqieiiments 
used  the  Mine  42  COCOMO  training  projects  and  21  COCOMO  testing  projeas.  In  the 
second  round  of  experiments  the  entire  63  project  COCOMO  project  set  was  used  for 
training  while  the  15  project  Kenunerer  data  set  was  used  as  the  testing  data.  The  training 
data  input  variables  were  KDSl,  the  annual  adjustment  fiictor,  and  the  fifteen  cost  drivers. 
The  actual  effort  associated  with  each  project  was  the  dqiendent  variable.  The  project 
mode  was  not  included  in  the  training  set  rince  nx>de  is  a  fixed  descr^tor  of  project 
classification  and  not  a  variable.  Consideration  was  given  to  partitioning  the  data  by  mode 
but  this  idea  was  discarded,  since  the  resulting  training  and  testing  sets  would  be  so  souH 
no  conclusions  could  reasorubly  be  drawn  from  the  results. 

The  third  step  in  the  genetic  programming  preparation  process  was  the  definition 
of  the  fimess  measure  embedded  in  the  "fitness,  c”  source  code  file.  With  the  knowledge 
gained  fium  the  sequence  of  genetic  algorithm  erqieriments,  the  fimess  measure  for  the 
genetic  programming  sequence  was  based  on  sv’erage  relative  error.  The  use  of  average 
relative  mot  is  also  supported  by  the  fact  that  some  initial  genetic  programmmg  test  runs 
peifrnmed  using  the  COCOMO  data  with  a  different  fimess  measure,  mean>squared  error, 
riiowed  very  poor  performance. 
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The  fourth  step  in  the  process  wts  the  preparation  of  the  test  data  that  SGPC 
used  to  test  the  resulting  S>e)q>ressions  during  the  training  process.  The  test  data 
prq>aration  consisted  of  umg  a  perl  scr^t  that  took  a  tab*delimhed  ASCD  file  containing 
the  test  data  and  built  a  C  source  code  file  that  contained  a  table  of  the  test  data  and  fire 
associated  table  declarations. 

The  fifth  and  final  stqr  of  the  genetic  programmmg  preparation  process  was  to 
coiqrfle  the  program  usmg  a  makefile  that  included  the  user  defined  training,  testing,  and 
fimess  source  code  files.  Once  the  compilation  was  complete,  the  training  process  was 
initiated. 

The  SGPC  program  required  that  a  number  of  parameters  associated  with  the 
genetic  programming  process  be  provided  by  the  user  m  a  defauh  file.  These  parameters 
include  the  population  size,  random  seed,  reproduction,  crossover  and  mutation 
parameters,  and  S-«q>ression  size  limits.  A  sample  defauh  file  is  diown  in  Rgure  3>4. 


■eed-112S7 
chedcpoint.fiequency  -S 
population_tize  «  2S00 
inax_depth_fior_iiew_traes  ■  3 
inax_d^h_after_crocsover-  10 
iiiax_inittant_depth  •>  4 
grow_iiiethod  •  RAMPED 
•elacttea_niethod  -  TOURNAMENT 
touniatnent_K  *  6 
crossover_ftinc_pt_fiactioe  *  0.2 
crosMrvtr_anyjNjfraction  «  0.2 
fititeK_jirop_repro_fiacttoii  *  0.2 
punmoayJSKHUx  *  0.01 


Figure  3-4  A  Sample  Genetic  Programming  Default  File 
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A  pq)uktion  size  of 2300  was  used  for  all  the  genetic  progranuning  experiments 
in  this  study.  Choice  of  this  population  size  was  basically  a  conq>roinise  between  the  need 
to  provide  a  large  population  to  ensure  genetic  divershy  and  the  memory  availabte  on  the 
Sun  workstatioiL  It  is  inqroitant  to  remeixd>er  that  during  portions  of  the  genetic 
programmmg  process,  two  copies  of  the  entire  population  are  stored  in  memory.  If  these 
populations  begin  to  contain  large  S*expressions,  the  memory  requirements  can  be  huge, 
eaaly  exceeding  SO  or  60  megabytes.  Tournament  selection  was  used  to  select 
S*e)q>ressions  for  the  reproduction  process  with  a  tournament  size  of  six.  The  paramony 
fictor  included  in  the  delauh  file  is  an  SGPC  variable  that  can  be  used  to  reward 
S>expres9ons  for  size  of  structure  as  well  as  performance,  whh  smaller  structures  being 
viewed  as  better  than  larger  ones  vriien  performance  is  equivalent.  While  the  use  of 
parsimony  is  optional  in  gaetic  programming,  it  was  used  in  the  COCOMO  tuns  at 
various  settings.  The  crossover  and  S-erqrression  size  parameters  were  also  varied  so  that 
a  total  of  seven  runs  using  the  COCOMO  training  and  testing  data  and  seven  tuns  uang 
the  COCOMO  and  Kemmerer  data  were  conqrleted.  The  resuhs  of  eadi  run,  which 
contained  the  best  structure  for  each  generation  and  hs  assodated  fitness,  were  written  to 
an  output  file. 

Once  the  testing  process  was  conqrleted  the  output  files  were  processed  by  first 
running  an  Ami  Pro  mao'o  on  the  output  file.  This  macro  substituted  the  Quattro  Pro 
spreadsheet  coordinates  for  eadi  of  the  input  variable  lumes.  Using  this  procedure 
allowed  the  best  structure  fi’om  eadi  output  file  to  be  pasted  directly  into  a  qrreadsheet 
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cdl,  «4iere  the  perforaumce  calculations  were  made  automatically  umg  a  set  of  linked 
qnreadsheets.  Hiis  action  greatly  reduced  the  processmg  time  required  for  each  output 
file. 
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rv.  ANALYSIS  OF  RESULTS 


A.  MEASURES  OF  PERFORMANCE 

llie  results  of  die  eiqieriments  for  all  three  machine  leaning  tedmiques  were  analyzed 

in  die  same  manner  to  determine  each  technique's  performance.  Three  measures  of 

performance  were  used  for  all  of  the  techniques.  These  three  measures  consisted  of  the 

average  magnitude  of  the  relative  error,  the  standard  deviation  of  the  magnitude  of  the 

rdative  error,  and  the  regression  coefficient,  R>squared,  of  the  estimated  man-months 

vmus  the  actual  man-months.  The  dioice  of  these  three  peifonnance  measures  was  based 

iqion  the  work  done  by  Kemmerer  (1987),  in  i^hidi  he  analyzed  the  performance  of 

aeveral  popular  software  cost  estimation  models  uang  a  15  project  data  set  that  he 

developed.  These  are  the  same  15  projects  that  were  used  as  one  of  the  sets  of  testing 

data  in  this  study.  The  use  of  the  magnitude  of  the  relative  error  is  intended  to  measure 

the  accura^  of  eadi  model  The  closer  this  measure  is  to  zero,  the  greater  is  the  accura^ 

of  the  model  The  R-squared  measure  gives  the  correlation  between  the  estimates  of  eadi 

modd  and  the  actual  project  results.  A  perfect  correlatimi  gives  an  R-squared  value  of 

one.  This  measure  is  meant  to  act  as  a  sort  of  "reality  chedi"  on  the  models^  ouqiuts  in 

seeing  wdiether  the  project  estimates  track  in  a  predictable  marmer  with  the  project  actuals. 

Since  eadi  machine  learning  mqierimental  procedure  involved  two  phases 

cormponding  to  different  software  devdopment  domains,  the  measures  of 

perftirmance  are  able  to  provide  insigbt  in  two  distinct  maimers,  ha  the  first  phase  of  each 
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eqperimeatal  procedure,  COCOMO  data  was  used  for  both  traming  and  testmg.  The 
performance  measures  for  each  of  the  methods  during  this  phase  gives  msight  into  whidi 
of  the  madiine  generated  modds  perform  best  vdien  c^erating  in  a  ^>edfied  developmeat 
domain.  Where  appropriate,  conq>arist»s  with  the  estimates  provided  by  Intermediate 
COCOMO  are  made.  In  the  second  phase  of  eadi  experimental  procedure,  the  same 
testing  data  set  Kemmerer  (1987)  used  when  he  made  his  study  of  several  popular  modds 
was  used  as  the  testing  data  set  for  the  machine  generated  models.  In  this  phase,  it  is 
posable  to  compare  the  estimating  capabilities  of  the  models  devdoped  by  the  three 
machine*leanuDg  techniques  with  several  well*imown  coa>estimatiQD  models  by  usmg  the 
resuhs  Kemmerer  obtamed  in  his  study.  These  models  include  SLIM,  ESTIMACS, 
Function  Pomts,  and  COCOMO.  This  conqsarison  is  useiul  because  it  gives  an  mdication 
of  the  ability  of  the  machine  generated  models  to  generalize  across  development  domains, 
whidi  is  critical  for  a  model,  or  modeling  tedmique,  to  be  successful 

The  machine  generated  models  were  also  analyzed  to  see  if  thdr  structure  could 
provide  insight  into  the  generd  relationshq)S  between  the  input  variables  and  the  modd 
ouq>uts.  This  analyas  was  performed  to  see  if  the  madime  generated  models  detected  any 
rdationdi^s  among  the  variables  that  may  have  gone  unnoticed. 

a  NEURAL  NETWORKS 

1.  First  Phase  Results 

The  initial  phase  of  neurd  network  training  involved  the  use  of  the  COCOMO 
data  set  for  both  training  and  testing.  Six  networks  were  trained  uang  42  projects  from 
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the  COCOMO  <Ui4i  set.  Testing  was  conducted  using  the  remaining  21  projects.  Table 
4-1  shows  the  performance  statistics  based  on  the  resuks  of  these  tests  fiv  the  six  best 
■etworitSw 


Table  4-1  Neural  Net  Performance  Results 
COCOMO  Tiaifiifig  and  Testing  Data 


Network  Ceafiguratioa 

AvgRel  Err 

(•A) 

StdDcvErr 

(%) 

R-sqoared 

(actvs.est.) 

sn 

91.19 

90.1? 

0.755 

t/9 

t9.t0 

16.26 

0.707 

11/10 

102.96 

126.06 

0.tl6 

16/19 

61.10 

6t.61 

0.726 

II/IS 

•136.t4 

272.6t 

0.753 

24/2S 

110.17 

109.93 

0.705 

Int  aXCN^O 

16.33 

15.76 

0.991 

XX/XX  -  Hidden  Layer  1/Hidden  Layer  2 


20  Neurons  —  1  Ou^wt  Neuron 


As  Table  4-1  indicates,  neural  networks  are  not  highly  accurate  in  their 
estimates,  but  their  estimates  do  correlate  relative^  good  to  the  actual  projea  efforts. 
Results  of  this  type  are  consistent  with  the  typical  expectations  of  a  neural  network  Ihe 
strength  of  neural  netwoiks  lies  in  their  ability  to  provide  generalized  outputs  oased  on 
tiieir  oputs.  ¥nm  this  perfective,  the  neural  nelworic  resuhs  from  the  first  phase  of  the 
eferiment  are  postive.  The  most  accurate  neural  network,  with  16  neurons  in  the  first 
hidden  layer  and  19  neurons  in  the  second  hidden  layer,  is  capable  of  providing  a  general 
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idea  of  the  magnftude  of  the  effon  required  for  a  particular  project  bong  devdqped  within 
the  aame  domain  in  whidi  the  network  was  trained.  Ihe  correlation  coeffident, 
R'Squared,  of 0.726  indicates  that  this  esdxnate  has  a  fairiy  strong  relationsh^  with  the 
actual  project  eflfort.  Why  this  particular  networit  performs  better  than  the  others  is  not 
mqilicitly  determinable.  In  general,  a  network  is  successful  when  h  has  just  enough 
neurons  to  provide  a  capability  for  feature  detection,  but  not  so  many  neurons  that  it  just 
"memorizes”  the  training  data.  While  not  as  accurate  as  the  algorithmic  modd. 
Intermediate  COCOMO,  a  network  has  the  advantage  of  requiring  less  analytical  skill  to 
develop  and  use,  and  it  probably  requires  less  development  time.  Based  on  the 
performance  characteristics  as  observed  in  Table  4-1,  the  best  use  of  a  neural  network 
would  probably  be  at  the  early  stage  of  projert  development,  where  a  manager  could 
perform  "what-iT  analyses  usmg  various  input  combinations  and  obtain  resuhs  very 
quickly  with  a  minimum  of  setup.  Additionally,  by  continually  retraining  the  model  with 
new  projects  the  manager  could  gain  further  advantages  smce  the  network  model  would 
be  continually  calibrating  itself  through  this  retraining  process.  If  the  organization  also 
uses  an  algorithmic  model  for  cost  estimation,  and  correlates  hs  estinoates  with  those  of 
the  neural  network,  a  sort  of  "early  warning  system"  that  could  indicate  when  recalibratiQn 
of  the  algorithmic  model  is  required  could  be  established.  A  significant  drop  in  the 
corrdation  between  the  two  estimation  methods  could  be  an  indicator  that  the  algorithmic 
model  is  in  need  of  cah*bration. 
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The  most  interesting  i^ect  of  this  phase  of  e)q;>erimentation  is  the  effect 
aetwoik  configuration  has  on  performance.  While  the  accuracy  of  the  estimate  varies  over 
a  range  of  qtproximatdy  70  percent,  the  standard  deviation  varies  over  a  range  of 
approximate  200  percent  These  results  indicate  that  %i^en  prqiaring  to  use  a  neural 
networic  a  good  strategy  would  be  to  begin  with  several  candidate  configurations  and 
continue  to  maintatw  gnd  compare  them  until  a  statistically  sound  analyss  can  be 
performed  to  detennme  the  best  configuration.  As  for  finding  the  best  configuration,  the 
BrainMaker  (1992)  thunib  rule  of  using  the  number  of  inputs  and  outputs  divided  by  two 
or  the  Eberhart  and  Dobbins  (1990)  rule  of  using  the  square  root  of  the  sum  of  the  inputs 
and  outputs  phis  a  couple  would  have  both  provided  networks  dose  to  the  ones  found 
ttsbg  the  brute  force  search.  However,  with  the  power  available  in  current  personal 
computers  the  BrainMaker  brute  force  seardi  is  not  an  uiueasonable  or  even  overly 

tmoo-consunung  approach  to  take  initially. 

2.  Second  Phase  Results 

In  this  phase  of  the  neural  network  eiqierimeotation  process  the  entire  63  project 
COCOMO  data  set  was  used  for  training  and  the  15  project  Kemmerei  data  set  was  used 
for  testing.  As  previoudy  stated,  the  network  configurations  from  the  first  phase  were 
retrained  to  die  optimal  performance  point  for  the  63  training  projects  and  then  tested. 

The  results  of  this  phase  of  tests  are  diown  in  Table  4>2. 

The  second  phase  results  using  the  Kemmerer  testing  are  very  interesting.  As 
can  be  seen  finm  Table  4*2,  the  top  performing  neural  networks  are  competitive  with,  and 
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Table  4>2  Neural  Net  Performaoee  Reeulti 
COCOMO  Tniniiig  and  Kemincitr  Testug  Data 


Nccwifc 

Caafignralioo 

AtfllclErr 

06) 

Sid  Dev  Err 

<%) 

■t-saiiared 

(Estvs.  Act) 

S/t 

103.24 

937.05 

0.557 

1/9 

1.040.57 

1.546.72 

0.4tt 

11/10 

32t.M 

555.49 

0.761 

16/19 

451.30 

6t3.10 

0.742 

It/lS 

621.00 

129.12 

0.651 

24/25 

33112 

691.62 

0.598 

SLIM* 

771.17 

661.33 

0.178 

lot  COCOMO* 

5t3.t2 

162.79 

0.599 

Function  Points* 

102.74- 

112.11 

0.553 

ESTlMACS*t 

t5.4t 

70.36 

0.134 

XX/XX  -  Hidden  Layer  1/Hidden  Layer  2 

*  (Kcmmerer.  1917,  pp.  422-425) 

t  ESTIMACS  statistics  based  on  9  of  15  progecu 

in  iome  cases  superior  to,  some  of  the  more  popular  cost>estimation  methods  in  both 
accuracy  and  correlation  fiictor.  The  best  network  m  this  phase  (1 1/10),  has  an  accuracy 
greater  than  that  of  either  SUM  or  Intermediate  COCOMO  and  a  correlation  fictor 
higbet  of  the  noodels  except  SUM.  While  none  of  the  networks  could  be 
considered  truly  accurate,  the  resuhs  of  this  e>q>eriment  indicate  that  networks  are  worth 
strong  consideration.  He  best  indication  that  networks  deserve  attmtion  can  be  seen  by 
ontsidering  the  resuhs  of  the  best  networks  with  those  of  the  Intermediate  COCOMO. 
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Both  the  networks  and  Intermedute  COCOMO  are  based  on  the  same  63  project  data  set, 
yet  the  best  networks  perform  agnificantly  better.  These  resuhs  seem  to  indicate  that  the 
top  networks  are  capable  of  capturing  some  level  of  meta-knowledge  about  the 
rdatkmdiips  between  the  cost-estimation  inputs  and  the  ouqmt. 

With  the  rdativdy  gmaH  amount  of  data  in  both  nuiid>er  of  hqruts  and  projects 
tiiat  was  used  for  training  these  networks,  the  levd  of  performance  obtained  is  gdll 
Bgnificant.  However,  the  sensitivity  to  configuration  is  once  again  apparent  AD  ax 
networks  learned"  the  63  projects  in  the  COCOMO  data  set  but  their  performance  on  the 
Kemmerer  data  ranges  greatly  in  both  accuracy  and  correlation.  This  divergence  in  the 
range  of  performance  resuhs  appears  to  to  magnified  by  the  change  of  development 
domains.  This  reinforces  the  suggestion  that  managers  diould  initially  maintain  nnilt^le 
network  configurations  until  some  level  of  confidence  is  obtained  as  to  which  network  is 

truly  the  top  pr.former. 

3*  Neural  Network  Structural  Analysis 

Although  neural  networks  are  relativdy  ea^  to  construct  and  operate,  it  is 

difBcuh  to  e)q}licitly  analyze  bow  they  are  processing  their  inputs.  One  method,  aiggeaed 
by  a  BrainMaker  user  and  poaed  on  Coiq)userve,  can  at  leaa  provide  the  user  with 
ina^t  into  whidi  inputs  affect  the  behavior  of  the  networit  the  greatea.  In  this  method 
the  fira  step  is  to  extract  the  conqilete  set  of  connection  weights  fi’om  the  BrainMaker 
network  definition  file  and  str^  the  bias  weights  oflf  of  each  layer.  The  second  aq)  is  to 
take  the  absolute  value  of  all  the  weights.  And  finally,  in  the  third  aq),  a  senes  of  matrix 
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nuh^lKatioiis  are  pafbnned,  bepmung  with  muhip^g  the  ouqiut  layer  cooaectkn 
wcaghtt  the  oooaecticMi  waists  to  the  second  hidden  layer,  lliis  process  is  rq>eated  in 
a  stepwise  manner  backward  through  all  of  die  hidden  layers  uttil  a  single  wdght  vector 
remains,  lliis  resulting  weight  vector  diows  the  magnitude  of  the  connection  wdg^ts  that 
each  aput  "sees”  through  the  network.  Examining  this  vector  can  give  some  indication  of 
vdiich  inputs  are  affecting  the  network  output  to  the  greatest  extent.  IhifottunateJy,  diese 
results  do  not  indicate  iKdiether  these  weights  affect  the  ouqiut  poddvely  or  n^adve^ 
once  tmly  the  magnitudes  are  used.  StiS,  this  approadb  is  useful  because  the  user  can  at 
least  see  'Mbat  inputs  the  network  considers  inqioitant.  Ibis  matrix  muh^fication  analysis 
was  conducted  on  three  of  the  networks  derived  in  the  sectmd  phase  of  the  neural  network 
mqieriinents.  Two  of  the  top  networks,  1 1/10  and  16/19,  along  with  the  worst  network, 
S/9,  were  examined  usmg  this  method.  The  top  10  connection  weight  magnitudes  for  the 
input  neurons  in  these  three  networks  are  shown  in  Table  4>3. 

The  resuhs  diown  in  Table  4>3  provide  some  mto’esting  inagbts  into  how  eadi 
of  these  networks  are  man^ulating  the  inputs  to  the  network  in  order  to  obtain  the  effort 
estimate.  For  exanqile,  the  two  best  networks  deeixgihagze  the  effect  the  size  of  the 
project  has  on  the  effort  estimate  whDe  the  worst  network  views  size  as  having  the  top 
effea  on  the  ou^nit.  The  cotEparison  of  how  these  networks  view  projea  size 
cncqisulates  the  argument  made  by  various  researchers  that  "fines  of  source  code"  is  in 
actuality  a  poor  predictor  of  project  effort  (Kemmerer,  1987,  pg.  418).  One  fiaor  that  all 
of  the  networks  examined  do  have  in  common  thou^  is  their  engihaas  on  projea 
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Table  4^  Top  Network-lnflueaciof  Input  Neurons 


1  Nctworii  11/10 

Nctworfc  16/19 

Network  8/9  | 

Itput 

Magnitude 

Input 

Magnitude 

liput 

Magnitude 

TURN 

266.21 

PCAP 

462.70 

LOG.KDSl 

24X06 

CPLX 

20t.23 

TURN 

437.40 

SD 

231.34 

mSM 

111.72 

STOR 

436.3t 

TURN 

203.07 

PCAP 

161.31 

SD 

391.33 

PCAP 

192.67 

ORG 

166.97 

CPLX 

3tt.t0 

AEXP 

179.77 

TIME 

163.14 

LOG.KDSl 

371.82 

E 

173.33 

ACAP 

139.61 

TIME 

366.30 

ORG 

169.01 

VIRT 

157.12 

AEXP 

363.64 

CPLX 

164.78 

LOG.KDSI 

149.13 

ORG 

363.12 

VIRT 

138.68 

VEXP 

141.42 

TOOL 

343.72 

Es3 

141.67 

complexity,  con^uter  mraaround  time,  and  programmer  capability,  lliis  emphasis  on 
project  complexity  and  programmer  capability  is  also  apparent  in  the  cost  driver  values 
riiat  Bodim  assigns  to  these  values  but  the  emphasis  that  the  networits  pbce  on  computer 
turnaround  time  is  not  reflected  in  the  values  that  Dodun  asagns  to  this  driver  (Bodim, 
1981,  pg.  118).  Looking  at  these  resuhs  from  the  other  direction,  the  emphaas  that 
Bodun  places  on  the  capability  of  the  analyst  is  not  reflected  in  the  analyas  of  the 
networks.  This  driver  only  qipears  once  in  Table  4>3,  and  is  ranked  only  seventh. 

^Vhfle  not  conduave  by  any  measure,  this  type  of  analysis  can  provide  a  project 
managCT  some  insight  into  tdiidi  network  inputs  are  affecting  the  output  to  the  greatea 
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dcfree  and  cao  serve  as  a  t)|>e  of  seositivity  analym  that  can  provide  indications  of  areas 
diat  may  posAly  require  a  hi^er  levd  of  attention. 

C  GENETIC  ALGORITHMS 
1.  First  Phtse  Result 

The  first  phase  of  the  genetic  algorithm  e)q>eriments  used  the  COCOMO 
training  and  testing  data  sets  and  Intermediate  COCOMO  as  a  modd  template.  In  this 
phase  18  diflferent  fimess  fimctions  were  tested.  These  fimess  fimctions  differed  based  on 
the  crossover  rate,  the  measure  by  which  the  error  was  detetmmed,  and  the  degree  to 
which  the  emstraints  on  the  values  generated  by  the  genetic  algorithm  were  applied. 

Table  4^  provides  a  summary  of  all  of  the  tests  conducted. 

As  previously  mentioned,  different  fitness  measures  were  tested  in  order  to  see 
which  fitness  measure  most  effectively  forced  the  genetic  algorithm  to  find  the  optimal 
values  of  the  cost  drivers,  coefBcients,  and  e^onents.  The  constraints  invoked  during  the 
various  tests  reflected  the  patterns  noted  in  Boehm's  values  for  the  cost  drivers.  The  ”no 
constramts”  tests  meant  that  afl  values  were  fiee  to  seek  optimal  values  independent  of 
eadi  other.  The  "cost  drivers  only”  constraint  pemalized  the  fitness  of  the  genetic 
algorithm  proportionate  to  the  degree  to  whidi  the  resuhs  ftfled  to  match  the  patterns 
noted  in  Bodun's  cost  driver  tables.  The  "all  constraints”  constraint  extended  the  "cost 
drivers  onV*  penalties  to  the  patterns  observed  in  the  coefiBdents  and  e)q>Qnents.  A 
gnq>hical  iq|)resentation  of  the  results  of  all  of  these  e)q>eriments  is  diown  in  Hgure  4-1. 
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Table  4-4  Genetic  Algorithm  Test  Series  Summary 
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OwMtie  AlgorW«n  Pvrfannane* 
eOCOMO  Tmwni  M  TmM^ 

400  1 - 


ToilNumbor 

IBSAob  fUlEff  MOMDtvimitl 


F^urc  4-1  Gcaetic  AlgMitbin  Accuracy  Performance 

Kgure  4-1  diows  that  the  best  petfonnance  in  the  genetic  algorithm  e)q>erimeats 
was  obtaiood  when  relative  error  was  used  as  the  fitness  measure  and  the  full  range  of 
constraints  were  applied.  These  performance  results  indicate  that  the  genetic  a^orithm  is 
an  effective  tool  for  detenmning  an  accurate  set  of  numerical  equivalents  fi»r  the  literal 
values  in  an  algorithmic  model  sudi  as  Intermediate  COCOMO.  The  correlation  fictors 
for  the  genetic  algorithm  derived  values  are  also  excellent ,  and  are  diown  m  Kgure  4-2. 

The  ability  diown  by  the  genetic  algorithm  to  derive  the  values  for  all  of  the  cost 
drivers,  coeffidents,  and  exponents  given  only  a  model  teiiq>late,  an  olgective  fimctkm 
lequiremeat  to  the  rdative  error,  and  some  constraints  diat  describe  knowledge 

of  the  domain  is  an  excellent  example  of  the  power  foat  this  madiine  learning  technique 
can  provide  to  the  manager. 
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Figure  4>2  Genetic  Algorithm  Correlation  Performance 

Bodun  (1981,  pg.  324)  describes  atuations  where  it  is  desirable  for  COCOMO 
to  be  taflored  to  a  particular  instalUtion.  The  procedure  outlined  for  calibratiDg  the  modd 
requires  solving  a  Qrtem  of  hnear  equations.  This  is  timooonsuming,  tedious  work  and  it 
also  fids  to  take  advantage  of  any  domain  knowledge  that  may  have  been  accumulated  by 
foe  organizaticm.  The  genetic  algorithm  provides  an  ahemative  method  to  this  q>proadi 
foat  is  easier,  more  extensible,  and  takes  advantage  of  any  domain  knowledge  foat  foe  user 
wifoes  to  embed  in  foe  fimess  fimction.  For  instance,  assumbg  a  COCOMO  template,  in 
foat  foe  effort  required  is  a  fimction  ofa  particular  cost  driver  set,  foe  project  Bze,  and  a 
particular  devdopment  mode,  a  user  could  eaafy  buOd  a  newpseudo>COCOMO  modd 
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Alt  makes  use  of  any  number  of  cost  drivers  and  modes  that  the  user  ^edfies  in  the 
fimess  function  and  duomosome  structure.  Additionally,  the  user  has  the  t^portunity  to 
build  into  the  fimess  fimction  any  identified  domam  knowledge  about  the  rdationdi9S  that 
are  applicable. 

2.  Second  Phase  Results 

In  the  second  phase  of  the  genetic  algorithm  experimentatioo  process  the  entire 
COCOMO  data  set  was  used  for  training  and  the  Kemmerer  data  set  was  used  for  testing. 
The  fitness  function  for  this  phase  used  the  magnitude  of  the  rebtive  error  and  the  full  set 
of  constraints,  taking  advantage  of  the  knowledge  gained  previously.  The  resuks  of  this 
e)q)etiment  are  riiown  in  Table  4-4. 


Table  4-4  Phase  2  Genetic  Algorithm  Performance 


COCOMO  Traming  and  Kemmerer  Testing  Data 

Model 

Av|.  Rel.  Error 

(%) 

Std.  Dev.  Error 

(%) 

R-squared 

CAlm.CCXX)MO 

131.01 

1,363.11 

0.472 

GA  Basic  COCOMO 

95.34 

111.75 

0.578 

Basic  COCOMO* 

■3H 

684.55 

0.680 

Int.  COCOMO* 

583.S2 

862.79 

0.599 

SLIM* 

771.17 

661.33 

0.878 

Function  Poiiitt* 

102.74 

112.11 

0.553 

ESTIMACS*  t 

t5.4t 

70.36 

0.134 

*  (Kenunerer.  19S7.PP.  422-42S 
t  ESTIMACS  statistia  base^  on  9  ef  15  pngectt 


Tlie  pafoimance  of  the  ge&edc  algorithni-derived  values  for  the  cost  drivos 
fired  pooriy  ^en  estmataig  the  effoit  required  for  the  Kemmerei  projects.  In  fict,  the 
OttTdatkn  coeflBdcat  finr  the  fitennediate  COCOMOmodd  derived  the  geaetic 
algorhhm  is  lower  for  the  Kemmerer  data  set  than  the  correlation  coefBcwnt  between  fie 
KDSI  and  fie  actual  project  inan*iDonths  for  this  data  set  (0.47  vs.  0.53X  indicating  that  in 
fiis  case  fie  hrtennediate  COCOM O  modd  adds  little  value.  Ihe  interesting  results  ii 
this  eiqperiment  are  obtained  by  uang  fie  Base  COCOMO  modd  ^ch  include;  just  fie 
coeffidents  and  erqionents  and  uses  no  cost  driver  values.  The  estimates  obtained  when 
using  fie  genetic  dgorithm  derived  values  for  fie  coeffidents  and  e}q>oneots  actually  give 
some  of  fie  best  resuhs  of  any  model  The  strongest  conclusion  that  can  be  made  from 
these  resuhs  is  that  fie  cost  driver  values  derived  by  fie  genetic  dgorithm  are  hi^ily 
senative  to  fie  domain  in  which  fie  model  was  trained.  These  resuhs  have  both  positive 
and  native  inqtacts.  On  fie  positive  ade,  they  reinforce  fie  conclusion  from  fie  fira 
phase  of  genetic  dgorithm  ejqteriments  that  fie  genetic  dgorithm  is  highly  effective  at 
learning  a  specific  software  devdopment  domain.  On  the  negative  side,  they  indicate  that 
any  resulting  modd  is  likely  to  be  of  little  benefit  if  there  is  a  dramatic  diift  in  fie 

underlying  fictors  in  fie  devdopment  environment. 

3.  Genetic  Algorithm  Structural  Analystt 

The  surpriang  accuracy  of  fie  Basic  COCOMO  modd  as  derived  by  fie  genetic 

dgorithm  is  fie  mod  interesting  resuh  of  this  e>q)erimeot.  A  posable  condusion  that  can 
be  drawn  from  this  resuh  is  fiat  fie  genetic  dgorithm  has  observed  a  different  relationsh^ 


between  the  coeflBcknts  and  e^aneots  of  the  COCOMO  oKidel  then  that  observed  by 
Bodua  Table  4-5  diows  the  genetic  algorithm  coefficients  and  exponent  values 
compared  to  those  devdoped  by  Bodua 


Table  4-5  Comparison  of  Cocflicientt  and  Exponents 


1  GcMtk  AlforithB  Valiict  vs.  Becba  VahMs 

Embedded 

Semidetached 

Organic 

CA  CoefficiaiMt 

0.429 

0.SS2 

2.25 

GA  Expments 

1.27 

1.21 

0.S10 

Boehm  Ceelficiciiu* 

2.S 

3.0 

3.2 

Boehm  Exponents* 

1.20 

1.12 

l.OS 

1  *  (Boehm,  1911.  pg.  117) 

This  conpaiisoo  between  the  genetic  algorithm  derived  values  and  Bodun's 
values  for  the  coefficients  and  exponents  indicates  that  the  genetic  algorithm  detects  a 
greater  disecononiy  of  scale  (erqionent  greater  than  one)  for  both  the  enibedded  and 
senudetadied  modes  than  Boehm  does.  Also,  the  genetic  algorithm  gives  a  mudi  greater 
econoiny  of  scale  (erq>onent  less  than  one)  to  the  organic  mode  than  Bodun  does.  The 
difference  in  coefficients  between  the  genetic  algorithm  and  Boehm  is  probably 
attributable  to  the  difference  in  size  of  the  typical  projects  that  are  devdoped  in  each 
mode.  Ihe  large  difference  seen  in  the  embedded  and  semidetadied  coefficients  between 
die  two  versons  is  more  than  likely  a  reflection  of  the  genetic  algorithm's  detection  of  a 
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UroBger  rdatkuishq)  between  the  diseconomy  of  scele  in  these  two  inodes  rather  than  the 
ielationdi9  between  the  scafing  effect  of  the  coefBdents 

The  coat  driver  values  derived  by  the  genetic  algorithm  also  diow  some 
noticeable  differences  from  those  derived  by  Bodun.  Table  4^  riiows  the  coA  driven. 


Table  4-6  Cost  Driver  Values  from  GA  and  Boehm*  (GA/Boehai) 


VLOW 

LOW 

NOM 

VHIGH 

1^2231 

RELY 

053/0.75 

0.72 /Ott 

0.95/1.00 

1.01/1.15 

1.38/1.4 

DATA 

0.77/0.94 

0.81  / 1.00 

1.06/1.08 

1.06/1.16 

CPLX 

079/070 

Ot0/0.t5 

1.04/1.00 

1.28/1.15 

1.39/1.30 

nmm 

1.06/1.00 

1.13/1.11 

1.45/1.30 

1.97/ 1.66  1 

STOR 

1.16/1.00 

1.50/1.06 

1.57/1.21 

VIRT 

0.13/0.17 

0.87/1.00 

0.88/1.15 

TURN 

0.96/0.17 

0.99/1.00 

1.13/1.07 

1.34/1.15 

ACAP 

1.65/1.19 

1.54/1.00 

1.15/0.86 

AEXP 

1.14/1.29 

1.19/1.13 

1.19/1.00 

0.83/0.91 

Boa 

PCAP 

2.00/1.42 

1.07/1.17 

1.07/1.00 

1.07/0.86 

0.76/0.70 

1.72/1.21 

1.24/1.10 

1.08/1.00 

1.08/0.9 

LEXP 

1.64/1.14 

1.56/1.07 

1.32/1.00 

1.25/0.95 

1.23/1.24 

0.99/1.10 

0.98  / 1.00 

0.93/0.91 

um 

TOOL 

1.97/1.24 

1.44/1.10 

1.32/1.00 

OB 

1.92/1.23 

1.56/1.08 

1.54/1.00 

2  00/1.04 

1.70/1.10 

1  •  (Boehm.  19Sl.|ig.l  It) 

The  cott  driver  table  shows  some  annlarities  in  the  values  derived  by  the  genetic 
algorithm  and  the  results  of  the  analyris  performed  on  the  neural  networks.  For  instance, 
the  genetic  algorithm  values  allow  conqrlexity,  programmer  capability,  application 
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experience,  and  conq;>iiter  turnaround  time  to  exert  a  greater  variation  on  the  effort 
adjustment  &ctor  than  Bodun's  values.  These  are  also  factors  that  the  top  neural 
networks  emphaazed.  This  concurrence  of  en^hasis  between  two  very  different  machine 
learning  paradigms  may  be  an  indication  that  greater  attention  should  be  paid  by  managers 
to  these  areas. 

D.  GENETIC  PROGRAMMING 

1.  First  Phase  Results 

The  first  phase  of  the  genetic  programming  experimats  used  the  COCOMO 
training  and  testing  data  sets.  Several  runs  usmg  the  genetic  programming  software  were 
made  uring  various  defauh  values  of  such  factors  as,  the  noaximum  depth  for  new  trees, 
the  maximum  depth  after  aossover,  the  crossover  factors,  and  parsimony.  The  parsimony 
factor  was  tried  at  various  levels  to  see  what  effect  h  had  on  both  S>expression  length  and 
generalization  characteristics  of  the  resuhing  expresaons.  The  population  size  was  2500 
for  all  runs,  and  the  total  number  of  generations  for  each  run  was  50.  For  each  run,  the 
fittea  structure  from  each  generation  was  saved  to  an  outfile  file.  The  fimess  measure 
used  in  these  erqreriments  was  the  average  magnitude  of  the  relative  error.  The  fitness  of 
both  the  training  and  testing  data  sets  were  computed  earib  generation. 

The  resuhs  of  this  phase  of  experiments  were  very  encouragmg  and  followed  a 
distinctive  pattern.  Initially,  the  bea  of  generation  fitness  for  both  the  testing  and  training 
data  were  roughly  of  the  same  magnitude.  As  each  run  progressed,  the  average  rdative 
error  fi>r  both  the  testing  and  training  data  tended  to  decline  until  a  point  was  readied 
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vAere  ■  divergence  wts  noticed.  At  this  point,  the  average  relative  error  of  the  trainmg 
data  continued  to  dedine  v^e  the  average  relative  error  of  the  testing  data  would  be^ 
to  dimb,  aomedmes  m  a  dranutic  numner.  It  was  very  apparent  from  this  pattern  that  the 
capability  of  genetic  programnung  to  provide  a  generalized  expression  is  reduced  as  it 
leams  the  training  data  to  a  greater  and  greater  degree.  This  behavior  is  eq>edally 
noticeable  in  the  second  phase  of  the  genetic  prograimnmg  experience  \dren  the 
COCOMO  and  Kermnerer  data  are  used. 

As  for  performance,  in  every  run  genetic  programming  was  able  to  learn  an 
e>q;>ression  for  the  estimated  effort  that  was  both  accurate  and  weD^correlated.  A 
smmnary  of  some  of  the  top  performers  for  the  first  phase  is  diown  in  Table  4>7. 


Tabic  4-7  Genetic  Programming  Results— Phase  One 


1  COCOMO  Training  and  Testing  Dau  | 

Test! 

Test  3 

Testa 

Tests 

Test  6 

Int  COCOMO 

AvgRelEn 

39.«9 

41.9 

42.37 

42.1 

45.32 

16.33 

StdDevEir 

36.5 

39.73 

41.06 

33.44 

40.26 

15.76 

R-Squared 

B!I 

0.71 

095 

0.69 

mem 

As  Table  4-7  indicates,  the  accura^  and  correlation  of  the  genetic  programnung 
derived  eq^resaons  are  worthy  of  strong  conaderation  by  any  software  manag*"  The 
only  major  hindrance  to  the  ea^  acceptance  of  genetic  programming  as  a  useful  technique 
comes  from  examining  the  ejqpressions  that  the  genetic  program  generates.  These 
expressions  can  be  both  intriguing  and  intiniidating  to  the  uninformed  user.  It  is  important 
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to  remember  v^eo  usmg  genedc  programmiDg  that  the  genetic  progrtm  does  not  have  the 
domain  Imovviedge  nor  the  pr^dicet  of  the  user.  This  means  that  the  genetic  program  is 
free  to  use  the  termmals  and  fimcdons  provided  by  the  user  in  any  way  that  the  genetic 
program  perceives  as  *fit*.  As  an  example,  Kgure  4>3  shows  one  of  the  expressions 
generated  by  the  genetic  program  during  this  phase  of  the  experiments. 

MMEST-((((MODP*KDSI)  +  ((TIME^CPLX)*  (KDSI-  LEXP ))) ^VIRT) 

'^ACAP)  ♦  (((ACAP*((STOR'CPLX)*RELY))+ (((ACAP*(PCAP*  (TIME*  ((  KDSl 
/(((((PCAP*(KDSI  ''STOR)WDATA+((KDSl  '^CED)*  SCED)))  '"VIRT)  •((DATA+  CKDSI 
•  SCED  ))  +KDSI))  +  ACAP))  «KKDSI  "«TOR)))))  +  ((( KDSl  •  VEXP  r  nME^VEXP))  - 

Figure  4-3  A  Sample  Genetic  Programming  Expression 

As  can  be  seen  from  Figure  4-3,  the  expressions  generated  by  the  genetic 
program  can  be  con^>Iex  and  somewhat  cryptic,  but  they  are  also  accurate.  The 
expression  shown  in  Figure  4-3  is  the  expresaon  used  to  obtain  the  Test  1  resuhs  diown 
in  Table  4-7.  As  Table  4-7  shows,  this  erqrression  has  an  average  relative  error  of 

approximately  4C  percent  and  an  R-squared  value  of  0.9. 

2.  Second  Phase  Results 

The  second  phase  of  the  genetic  pro£:^amming  mqperiments  used  the  COCOMO 
data  for  training  and  the  Kemmerer  data  for  testing.  The  same  variations  in  program 
parameters  such  as  maximum  tree  depth,  crossover  rates,  parsimony,  etc.,  were  used  in 
this  phase  as  in  the  first  phase.  The  same  patten  in  the  fitness  measures  for  both  the 
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training  and  testing  data  were  also  observed  but  in  a  much  more  pronounced  muner.  In 
&ct,  the  divergence  in  fitness  between  the  training  and  the  testing  data  was  so  pronounced 
diat  the  maximum  number  of  generations  was  reduced  from  SO  to  20  for  this  phase  of 
e>q>eriments.  A  summaiy  of  some  of  the  top  runs  for  this  phase  of  experiments  is  diown 
in  Table  4-S.  The  results  from  Kemmerer's  analyas  are  included  for  corxpaiison. 


Table  4-8  Genetic  Programming  Performance  Results 
COCOMO  Training  and  Kemmcfer  Testing  Data 


Test 

Avg  Rcl  Err 

StdDcvErr 

R-squared 

(%) 

(%) 

(Estvs.  Aet) 

Tests 

109.52 

125.85 

0.78 

Testa 

130.f7 

142.35 

0.62 

Testa 

59.15 

69.5 

0.93 

Test  9(a) 

112.01 

132.15 

0.90 

Test  9(b) 

144.24 

167.9 

0.85 

SLIM* 

771.87 

661.33 

0.878 

IntCOCCK^O* 

583.82 

862.79 

0.599 

Function  Points* 

102.74 

112.11 

0.553 

ESTIMACS*t 

85.48 

70.36 

0.134 

*  (Kenunerer,  1987.  pp.  422-425) 

t  ESTIMACS  statistics  based  on  9  of  IS  prefects 

As  Table  4-8  indicates,  genetic  programming  is  very  capable  of  deriving 
erqiressions  that  can  provide  higbly  accurate  results  across  development  domains.  An 
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cample  of  the  type  of  e)q>reeaon  geoented  by  the  genetic  program  during  this  phase  is 
diowB  in  Figiire  4-4. 


MMEST«(((TOOL>  VEXP)»  RELY)-f  (((KDSI*  ACAP)''  VIRT)'' 
(MODP*  TIME)))-^  (KDSI^  STOR) 

Figure  4-4  A  Sample  Genetic  Programming  Eapression 

The  erfpresaon  diown  in  Figure  4-4  is  the  expression  used  to  obtain  the  results 
in  Test  9(a)  diown  in  Table  4-8.  The  resuks  obtained  from  this  e)q>ressioo  are  atpaior  to 
most  of  the  models  tested  by  Kemmerer  (1987)  with  the  exception  of  ESTIMACS. 
However,  one  of  the  eiqiressions  generated  by  the  genetic  program  during  this  phase  is 
even  superior  to  ESTIMACS.  This  mq)ression  is  the  top  performer  from  several  of  the 
runs,  and  is  the  same  expression  used  to  obtain  the  resuhs  diown  in  Test  4  of  Table  4-8. 
The  erqtresaon  is  diown  in  Figure  4-S. 


MM  EST  -  (  KDSI  +  (2.336426)  )  ^  STOR 

Figure  4-5  Top  Performing  Expression  from  Phase  2 
This  mqiresaon  is  extreme^  ample  and  yet  higU^  accurate,  as  the  results  in 
Table  4-8  indicate,  yet  is  highly  unlikely  that  a  human  domain  expat  would  conceive  of  an 
expresdott  Sttdi  as  this  one.  This  is  where  genetic  programming  diows  its  usefulness.  The 
ability  to  concave  of  relationdiips  among  various  frctors  without  the  pr^dice  of  one's 
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own  know^ge  is  what  allows  the  genetic  program  to  succeed  in  its  task  The  results 
from  this  phase  of  e>q>eriments  indicate  that  genetic  programmmg  is  an  extremely  usesfol 
tedmique  for  generating  cost-estimation  models  and  diould  be  considered  as  a  leading 
candidate  for  use  as  a  cost-estimation  model  generator. 

The  coiig>arison  of  the  a  sanq)le  of  the  resuks  of  genetic  programming  with 
those  obtained  by  Kemmerer  in  his  study  shows  that  genetic  programming  is  capable  of 
gmerating  a  model  that  is  superior  to  any  of  these  popular  cost-estimation  models,  in 
fret,  iq>on  examining  the  best  models  from  each  generation  of  all  of  the  runs  during  this 
phase  diowed  that  during  the  20  generation  cycle  for  each  run,  the  genetic  program  rarely 
generated  a  program  with  over  400%  en  or.  This  means  that  at  almost  any  time  during 
any  run,  the  genetic  program  was  generating  expresaons  that  were  superior  to  both  SLIM 
and  intermediate  COCOMO. 

3.  Genetic  Program  Structural  Analysis 

The  ana^^  of  the  structures  generated  by  the  genetic  program  is  a  very 

conqtlex  topic  and  is  currently  being  studied  by  a  variety  of  researchers  worldwide. 
Ahhou^  researchers  are  of  different  opinions  about  the  significance  of  the  structures  that 
genetic  progranuning  produces,  there  are  some  baac  characteristics  that  can  be  observed. 

Aiudyzing  the  resuhs  of  a  genetic  progranuning  run  shows  that  the  genetic 
progranunmg  paradigm  can  be  very  inventive.  As  an  example,  when  Koza  (1992,  pg.  242) 
first  developed  his  LISP  genetic  programming  software,  the  capability  to  generate 
constant  terms  was  not  inchided.  During  an  early  run  bvolving  the  discovery  of 
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trigonometric  identities  he  observed  t  peculiar  term  in  the  resulting  LISP  S-expression. 
This  term  is  shown  in  Figure  4*5. 

2  -  Sin(Sm(Sin(Sin(Sin(Sin(Sin(Sin(l))*Sin(Sin(l ))))))))) 

Figure  4>S  Genetic  Programming  Derived  Constant 

This  term  evaluates  to  1.57,  whidt  is  very  close  to  the  value  of  Pi  divided  by 
two,  a  constant  that  is  needed  often  in  trigonometric  problems.  The  discovery  of  this  type 
of  inventive  behavior  by  the  genetic  program  led  Koza  to  modify  the  software  to  allow  for 
the  creation  of  random  constants.  Koza  (1992,  pg.  1 85)  also  observed  a  tendency  for  the 
genetic  program  to  take  useful  subtrees  and  repetitively  use  them  to  perform  functions  that 
the  genetic  program  found  useful 

The  structures  derived  by  the  genetic  program  for  the  second  phase  in  these 
experiments  varied  widely  in  their  size  and  composition,  although  the  use  of  parsimony 
and  a  reduced  maximum  tree  depth  tended  to  yield  shorter  structures  that  performed 
better  across  projea  domains.  Examples  of  some  of  the  structures  derived  by  the  genetic 
program  for  cost-estimation  are  shown  in  Figure  4-6. 

Examining  the  structures  in  Figure  4-6,  it  is  apparent  that  the  genetic  program 
does  not  find  all  of  the  cost  drivers  useful  While  KDSL  MODP,  STOR,  and  TIME 
appear  with  some  regularity,  other  drivers  sudi  as  TURN  and  SCED  are  not  used  at  all 
This  is  where  genetic  programmiDg  can  be  unsettling  to  a  user.  While  the  user  can  give 
copious  amounts  of  data  to  the  genetic  program,  there  is  no  certainty  that  all  of  it  will  be 


Test  2  GcBcratwB  6 

MMEST-((AAF''PCAP)+  LEXI*)+  (((CPLX+  ((((1.615063)  •  KDSI)* 
((DATA*  TIME)*  TC«L))+  ((KDSI''TIME)^PCAP)))''VIRT)-  VIRT) 

Valutadon  Fitneiv*  109.31 
Tctt3  GcBcratioa  6 
MMEST-KDSl*  (AEXP+  TIME) 

Validation  Fitness  130.77 
Teat  9  Gcacratioa  6 

MM  EST  -  (TCXX,  +  ((KDSl  (MC»)P  •  TIME))  +  (KDSl  STC»)))  *  VIRT 
Validation  Fitness*  144.16 
Test  9  GcncratioB  10 

MM  EST  » (((TOOL  +  VEXP)  •  RELY)  +  (((KDSl  •  AC AP)  VIRT)  (MODP  • 
TIME)))+  (KDSI''STOR) 

Validation  Fitness*  1 12.01 


Figure  4-6  Genetic  Programming  Derived  Structures 


used.  Hie  user  has  to  be  prepared  to  deal  with  structures  that  have  no  resemblence  to 
relationsh^s  that  the  user  bdieves  to  enst  within  the  domain.  This  faaor  may  be  a  barrier 
to  accqitance  for  some  managers,  but  if  the  models  derived  are  statistically  significant  in 
both  accuracy  and  correlation  they  are  hrrd  to  dismiss. 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  CONCLUSIONS 

Die  major  otgective  of  this  thesis  was  to  examme  the  effecdvesess  of  three  machine 
learning  techniques  as  applied  to  the  field  of  software  cost  estimation.  Die  driving  ftctor 
behind  the  mvesdgatkm  of  madune  learning  in  the  cost  estimation  fidd  is  that  current 
software  cost  estimation  models  are  both  inaccurate  and  not  very  adaptable  across 
development  domains.  Diese  are  critical  areas  of  weakness  smce  the  Department  of 
Defense  is  the  largest  single  software  consumer  in  the  world,  and  it  develops  software  for 
a  wide  variety  of  domains.  Dierefore,  h  is  necessary  that  a  software  cost  estimation 
model  be  both  accurate  and  eaaly  adaptable  in  order  to  be  considered  successful  Die 
premise  of  this  research  was  that  there  probably  were  relationsh^s  among  the  various 
software  project  &ctors  that  the  current  models  did  not  capture  and  that  a  machine 
learning  approach,  where  the  models  are  derived  directly  from  the  data,  may  prove 
superior  in  both  performance  and  adaptability. 

Die  three  machine  learning  techniques  studied  in  this  thesis  have  all  proven  through 
the  eiqierimaits  conducted  to  have  great  usefulness  in  the  area  of  software  cost 
estimation.  Each  tedinique  studied  also  brings  to  this  field  certain  particular  capabilities. 

Neural  networks  diowed  a  strong  capability  to  capture  and  learn  the  project  data,  and 
make  estimates  of  reasonable  accuracy  that  exhibited  a  high  correlation  with  the  project 
actuals.  Whoi  tested  with  data  from  outride  the  domain  in  wfaidi  the  networks  were 
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devdoped,  the  oeunl  networics  diowed  •  strong  capability  to  generalize  on  the  data  that 
was  presented  to  them  and  make  estimates  that  were  siq>erior  to  those  of  other 
weD^lmown  naodds.  This  capability  to  make  estimates  across  domains  is  oticd  when 
seeking  estimates  on  new  projects  with  different  development  envirmunents.  Given  the 
limited  amount  of  data  that  was  available  for  testing  the  networks,  they  perfonned 
suipiisingly  wdL  Neurd  networks  also  have  an  advantage  in  that  they  are  relativdy  ea^ 
to  set  up  and  train,  and  are  ceitainly  within  the  capabilities  of  most  management  personod. 
They  also  have  the  advantage  of  being  self-calibrating,  as  long  as  the  netwoik  is 
maintained  by  continiuDy  retraining  with  data  from  newly  coiq;>leted  projects.  While  not 
as  accurate  as  some  of  the  algorithmic  models,  neural  networks  are  certainly  worthy  of 
consideration,  e^ecially  since  they  have  the  ability  to  ded  with  non-numeric,  or  ^ndmUc, 
data.  The  best  use  of  a  neural  network  wHhin  an  organization  is  probab^'  et  the  eariiest 
stages  tdiere  estimates  that  can  at  least  capture  the  likely  naagnhude  of  a  project  are  the 
most  useful 

The  genetic  dgorithm  experiments  showed  that  they  were  higmy  effective  as  a  tool 
for  model  optimization  and  cah'bration.  Given  no  more  than  a  fimess  function,  a  table  of 
fiterd  values  describing  the  projeas,  and  a  small  amount  of  domain  kno^dedge  in  the  guise 
of  constraints  on  the  fimess  fimction,  the  genetic  dgorithm  was  able  to  find  a  conq)lete  set 
of  values  for  the  cost  drivers,  coefficients,  and  e)q)onents  in  the  Intermediate  COCOMO 
modd.  The  resulting  model  values  were  shown  to  provide  estimates  that  had  a  high 
degree  of  accuraQ^  and  correbtion  for  projects  developed  within  the  same  domain,  but 
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their  paformance  wts  not  as  good  when  applied  to  projects  outside  of  the  domain  in 
^^ikh  they  were  trained.  StiD,  the  genetic  algorithm  proved  to  be  effective  as  a  tool  for 
modd  calibration  and  it  is  ea^  to  apply.  The  user  has  only  to  supply  a  fitness  fimctkm 
that  inchides  the  &ness  measure,  the  model  fimction  and  whatever  constraints  the  user 
ftels  necessary  for  the  particular  domain.  The  power  of  evolution  handles  the  seardi  for 
an  optimal  solution.  Additionally,  the  genetic  algorithm  is  highly  extensible  due  to  the 
ability  of  the  chromosome  structure  to  be  modified  to  reflect  additional  characteristics  of 
the  domain.  The  genetic  algorithm  is  likely  to  be  of  the  greatest  use  m  a  situation  where  an 
organization  feels  they  have  a  model  for  how  their  development  process  functions  but  they 
need  a  method  for  calibrating  the  model  to  the  data  at  hand.  The  genetic  algorithm  is  very 
capable  in  this  situation. 

In  the  third  set  of  eiqietiments  the  very  new  technique  of  genetic  programmmg  was 
used  as  a  means  for  model  discovery.  In  this  case,  the  only  mputs  provided  were  the 
fimess  measure,  the  project  data,  and  a  set  of  functions  that  were  allowed  to  operate  on 
the  data.  Upon  initializing  the  population  with  a  random  set  of  programs,  the  process  of 
evolution  then  generated  models  of  greater  and  greater  fimess  as  the  genetic  program 
evolved  through  succeeding  goierations.  The  models  resulting  from  this  tedmique  were 
both  highly  accurate  and  strongly  conelated.  These  resuhs  indicate  that  genetic 
programming  diows  great  usefulness  as  an  exciting  new  approach  to  the  cost  estimation 
problem  and  has  probabfy  the  broadest  range  of  uses  of  any  of  the  tedmiques  studied  in 
thistheas. 
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Li  all  instances,  ead>  of  the  nwthods  examined  in  this  thesis  was  capable  of  generating 
models  that  were  conpetitive  in  both  accuracy  and  correlation  with  other  establidied  cost 
estimation  models,  ha  one  case,  the  machine  generated  model  outpoformed  all  of  the 
more  well>known  and  established  models.  This  level  of  performance  exhibited  by  madiine 
generated  models  indicates  that  they  should  be  given  serious  consideration  by  those 
involved  in  the  software  development  field,  as  they  offer  several  advantages.  First,  they 
directly  reflect  the  data  they  are  trained  whh,  wfaidi  automatically  ^es  them  an  advantage 
over  a  models  that  may  have  been  developed  outside  of  the  domain  b  which  it  is  bemg 
qtplied.  Second,  they  are  highly  extensible,  meanbg  that  any  additional  knowledge  that 
becomes  available  regardbg  the  software  development  process  can  easily  be  bcoiporated 
b  these  models.  This  provides  an  advantage  over  the  more  traditional  models  b  that  a 
madibe  generated  model  is  easily  tailored  to  the  data  available.  These  advantages, 
coiq>led  with  the  performance  levels  noted  b  the  erqjeiiments  conducted  b  this  study 
bdicate  that  madibe  leainbg  is  an  effective  and  useful  technique  for  the  field  of  software 
cost  estimation. 

B.  RECOMMENDATIONS  FOR  FURTHER  RESEARCH 

There  are  several  areas  that  are  candidates  for  fiuther  researdb  based  on  this  study. 

Hrst,  additional  experiments  can  be  conducted  to  extend  the  level  of  knowledge  regardbg 

how  each  of  the  techniques  diould  be  applied  to  adiieve  the  greatest  effectiveness. 

Secondly,  researdi  can  be  conducted  bto  couplmg  these  madibe  leatnbg  tedmiques 

together  b  order  to  neate  a  suite  of  tools  that  may  be  able  to  provide  a  siqierior  level  of 
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perfonnnice.  As  an  example,  one  could  bufld  an  expert  system  front  aid  that  could  query 
a  usa  about  project  characteristics  in  order  to  populate  a  data  base.  This  expert  system 
frtmt  end  could  then  be  coupled  to  a  neural  network  that  could  provide  general  estimates, 
or  it  could  be  coupled  to  the  genetic  program,  whidi  could  generate  candidate  modds.  It 
may  also  be  posable  to  have  a  neural  network  monitor  the  resuhs  of  the  models  the 
genetic  program  develops,  and  make  a  selection  as  to  which  model  is  bea  for  a  particular 
situation  based  on  the  user's  inputs  the  expert>syaem  front'cnd.  Finally,  a  sa  of 
experiments  to  those  conducted  in  this  study  could  be  performed  usmg  data  from  a  variety 
of  domains  for  training.  This  may  prove  maght&l,  ance  a  certain  amount  of 
meta>knowledge  about  the  data  would  have  to  be  apparent  in  any  models  developed  using 
these  techniques. 
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APPENDIX  A 


COCOMO  DATA  SET  (Boehm,  1981,  pg.  496) 
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APPENDIX  B 


COCOMO  Cost  Driver  Table  (Boehm,  1981,  pg.  118) 
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APPENDIX  C 


Kcmmerer  Data  Set 


ra 

ca 

K 

A 

R 

D 

C 

T 

S 

V 

T 

A 

A 

P 

V 

L 

UM 

m 

cm 

r 

D 

A 

B 

A 

P 

1 

T 

1 

U 

C 

B 

C 

B 

B 

O 

• 

A 

t 

r 

L 

T 

L 

M 

O 

R 

R 

A 

X 

A 

X 

X 

M 

1 

C 

I 

• 

Y 

A 

X 

E 

R 

T 

14 

P 

P 

P 

P 

P 

p 

[Q 

» 

T 

■ 

■ 

c 

1 

■ 

■ 

1 

1 

3«7 

333.6 

1 

1.13 

1.08 

1.13 

1 

1.06 

o.n 

1 

0.86 

1.13 

0.86 

1 

■ 

0.91 

20 

Ql 

1 

US 

40.3 

1 

1 

1 

1 

1 

1 

0.87 

1 

1 

1.07 

■ 

■ 

1 

■ 

■ 

SD 

I 

1.107. 

430 

1 

1 

1.16 

1 

1.3 

1.31 

0.87 

1 

0.86 

1.13 

1 

1 

1.03 

0.91 

■ 

1 

E 

1 

31 

■ 

4 

86.9 

314.4 

1 

1 

1.08 

0.93 

1 

1 

0.93 

0.8 

0.89 

0.83 

23 

1 

1 

02 

SD 

S 

336.3 

449.9 

B 

■ 

IQI 

1 

1 

1 

0.87 

0.78 

1.13 

0.93 

n 

IQ 

1 

0.91 

20 

ORG 

6 

84 

so 

■ 

■ 

1.16 

1 

1.11 

1 

0.87 

0.86 

1.13 

0.86 

QQ 

0.91 

IQ 

E 

7 

U.3 

43 

1 

1.16 

1 

1.11 

B3 

1 

0.87 

0.86 

1.13 

0.7 

m 

02 

0.91 

Q 

E 

> 

130.3 

167 

I.I3 

1.16 

1 

1 

1 

0.93 

1 

0.93 

1 

0.93 

Qg 

1 

QQ 

□1 

1 

116 

a 

B 

1.16 

1.13 

Ml 

. 

1.06 

0.87 

0.87 

1 

1.13 

1.06 

1.1 

Bl 

■ 

1.08 

E 

0 

73 

39 

1 

1.16 

> 

1.11 

M 

1.03 

0.71 

1 

0.7 

1.1 

QQ 

1.03 

^0 

SD 

n 

358.7 

354.3 

1 

1.16 

■ 

■ 

1 

0.86 

1.13 

0.86 

■ 

■ 

0.86 

00 

SD 

i 

330.7 

138.6 

1 

1.16 

1 

1 

0.87 

0.86 

1.07 

0.86 

■ 

D 

1.1 

0.95 

IQ 

SD 

Q! 

137 

161.4 

1 

1.16 

1 

1.06 

QQi 

0.87 

1.19 

1 

0.78 

Qj 

021 

1.1 

1 

IQI 

SD 

a> 

346.9 

164.8 

Bl 

■1 

1.16 

1 

1 

0.87 

0.87 

0.86 

1.31 

0.93 

1.1 

ni 

0.93 

IQII 

ORG 

Bl 

69.9 

60.3 

Bl 

■1 

■1 

1 

1 

1 

0.87 

0.71 

0.91 

0.86 

1.1 

IQI 

0.91 

0.91 

□ 

SD 

*  AAF  vduci  MUBid,  ao  vahuc  |iv«  by  Kamnr 


□ 

a 

n 

n 

D 

D 

D 

n 

03 

CQ 

03 

^3 

n 

n 

D 

n 

D 

23 

□ 

D 

SJ 

a 

^3 

D 

D 

n 

03 

^3 

D 

03 

a 

Q 

[Q 

gg 

Q| 

D 

*33 

43 

45 

45J 

03 

QO 

D 

03 

03 

IQ 

Q 

D 

m 

03 

03 

IQ 

gg 

B 

□ 

a 

as.6 

D 

^3 

IQ 

D 

m 

^3 

03 

Q 

D 

n 

m 

03 

D 

o 

03 

33 

n 

30.6 

03 

^3 

03 

Q 

03 

03 

a 

D 

a 

m 

03 

B 

D 

IQ 

23! 

□ 

la 

35 

ta 

Q] 

D 

Q 

03 

^3 

o 

D 

a 

m 

03 

D 

gg 

03 

33 

136 

73 

fsa 

BS 

Q 

D 

Q 

03 

00 

n 

D 

m 

03 

D 

a 

Q 

23j 

□ 

36 

33 

a 

03 

Q 

□ 

n 

D 

^3 

03 

Q 

m 

n 

D 

a 

03 

SSj 

□ 

1^73 

464 

Qj 

ED 

D 

D 

03 

a 

IQ 

03 

m 

D 

D 

IQ 

□ 

□ 

156 

91 

n 

D 

n 

Q 

D 

n 

D 

Q| 

oo 

D 

m 

n 

a 

03 

IQ 

D 

SO 

176 

34 

n 

IQ 

a 

D 

D 

IQ 

D 

B3 

Q 

n 

IQ 

n 

a 

n 

D 

D 

□ 

□ 

133 

10 

D 

IQ 

a 

a 

D 

D 

a 

IQ 

Q 

D 

Q 

n 

^3 

gg 

B 

OQ 

Q3| 

□ 

14 

5.3 

n 

^3 

Q 

Q 

D 

D 

Q 

n 

m 

n 

D 

B 

03 

□ 

D 

30 

D 

n 

D 

Q 

n 

D 

03 

IQ 

D 

a 

gg 

n 

a 

EB 

B 

a 

5Q 

m 

IS 

6.3 

D 

03 

Qj 

a 

a 

D 

03 

03 

Q 

Q 

BO 

gg 

B 

D 

D 

[Q 

□ 

37 

D 

ns 

Q 

D 

□ 

Q 

D 

D 

Q 

Q 

D 

D 

B 

B 

IQ 

B 

□ 

337 

17 

D 

03 

m 

^0 

^3 

D 

a 

D 

n 

n 

D 

^3 

B 

B 

B 

B 

□ 

130 

35 

D 

n 

001 

D 

OB 

BO 

D 

a 

gg 

go 

Q 

Q 

03 

03 

a 

D 

B 

□ 

D 

9.1 

n 

B3 

D 

QQ 

BO 

gg 

a 

GB 

BO 

m 

gg 

gg 

03 

gg 

n 

Q 

15 

10 

n 

n 

OQ 

IQ 

a 

D 

D 

OQ 

gg 

gg 

Qj 

a 

n 

03 

D 

D 

B 

95 


COCOMO  TESTING  DATA  SET 
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APPENDIX  E 


BrainMaker  Neural  Network  Definition  Files  (*.DEf) 
COCOMO  Training/Kemmerer  Testing  Version 


Network  S/8 

iq)ut  number  1  20 

dictionary  input  LOG_KDSI  AAF  RELY  DATA  CPLX  TIME  STOR  VIRT  TURN 

ACAP  AEXP  PCAP  VEXP  LEXP  MODP  TOOL  SCED  E  SD  ORG 

output  number  1  1 

dicdonaiy  output  LOG_MMAC 

hidden  S  8 

filename  trainfacts  c:\braiD\test\cokealll.fct 
filename  runfacts  c:\brain\test\kemmer.in 
feamrate  0.9000  50  0.75  75  O.OOOO  90  0.5000 
leamlayer  1.0000  1.0000  1.0000 
traintol  0.1000  0.04  0.8000  100 
testtol  0.2000 
random  5.0 
maxiuns  459 

fimction  hidden]  sigmoid  0.0000  1.0000  0.0000  1.00000 

fimction  hidden2  sigmoid  0.0000  1.0000  0.0000  1.00000 

fimction  output  agmoid  0.0000  1.0000  0.0000  1.09000 

LOG.KDS  AAF  RELY  DATA  CPLX  TIME  STOR  VIRT  TURN 

ACAP  AEXP  PCAP  VEXP  LEXP  MODP  TOOL  SCED  E  SD 

ORG 

LOG^MMA 
scale  input  minimum 


0.29666  0.43  0.75 
0.95  0.82  0.83 
scale  input  maximum 

0.94  0.7  1 

1  000 

1  0.87 

0.87 

0.71 

0.82 

0.7 

0.9 

3.06069  1  1.4 

1.14  1.24  1.24 

1.16  1.65  1.66 

1.23  1  1  1 

1.56  1.3 

1.15 

1.46 

1.29 

1.42 

1.21 

scale  ouqrut  minimum 
0.77085 


scale  ou^ut  maximum 
4.0569 
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Network  8/9 


n^ut  number  I  20 

dicUonaiy  input  LOG_KDSI  AAF  RELY  DATA  CPLX  TIME  STOR  VIRT  TURN 

ACAP  AEXP  PCAP  VEXP  LEXP  MODP  TOOL  SCED  E  SD  ORG 

output  nund>er  1  1 

dicdonaiy  output  LOG  MMAC 

hidden  8  9 

fQenune  tnin&cts  c:\bram\test\cokeaIll.fct 
filename  nin&cts  c:\brain\test\kemmer.in 
leamrate  0.9000  50  0.75  75  0.6000  90  0.5000 
leainlayer  1.0000  1.0000  1.0000 
trainto]  0.1000  0.04  0.8000  100 
testtol  0.2000 
random  5.0 
maxruns  430 

fimcdon  hidden  1  sigmoid  0.0000  1.0000  0.0000  1.00000 

fimction  hidden2  sigmoid  0.0000  1.0000  0.0000  1.00000 

fimction  output  sigmoid  0.0000  1.0000  0.0000  1.00000 

LOG  KDS  AAF  RELY  DATA  CPLX  TIME  STOR  VIRT  TURN 

ACAP  AEXP  PCAP  VEXP  LEXP  MODP  TOOL  SCED  E  SD 

ORG 

LOG_MMA 
scale  input  mminaum 


0.29666  0.43  0.75 
0.95  0.82  0.83 
scale  input  maidmum 

0.94  0.7  1 

1  000 

1  0.87 

0.87 

0.71 

0.82 

0.7 

0.9 

3.06069  1  1.4 

1.14  1.24  1.24 

1.16  1.65  1.66 

1.23  1  1  1 

1.56  1.3 

1.15 

1.46 

1.29 

1.42 

1.21 

scale  output  minimum 
0.77085 

scale  output  maximum 
4.0569 
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Network  11/10 


■q>iit  aiiiid>er  1  20 

dictioiuiy  input  LOG  KDSI  AAF  RELY  DATA  CPLX  TIME  STOR  VIRT  TURN 

ACAP  AEXP  PCAP'VEXP  LEXP  MODP  TOOL  SCED  E  SD  ORG 

output  number  1  1 

dictionary  output  LOG_MMAC 

hidden  11  10 

filename  trainfacts  c;\brain\test\cokealIl.fct 
filename  runfacts  c:\brain\test\kemnaer.m 
leamrate  0.9000  SO  0.7S  75  0.6000  90  O.SOOO 
leamlayer  1.0000  1.0000  1.0000 
traintol  0.1000  0.04  0.8000  100 
testtol  0.2000 
random  S.O 
maxruns  250 

fimction  hidden  1  sigmoid  0.0000  1.0000  0.0000  1.00000 

fimction  hidden2  sigmoid  0.0000  1.0000  0.0000  1.00000 

fimction  output  Bgmoid  0.0000  1.0000  0.0000  1.00000 

LOG  KDS  AAF  RELY  DATA  CPLX  TIME  STOR  VIRT  TURN 

ACAP  AEXP  PCAP  VEXP  LEXP  MODP  TOOL  SCED  E  SD 

ORG 

LOG_MMA 
scale  input  mininium 


0.29666  0.43  0.75 
0.95  0.82  0.83 
scale  input  maximum 

0.94  0.7  1 

1  000 

1 

0.87 

0.87 

0.71 

0.82 

0.7 

0.9 

3.06069  1  1.4 

1.14  1.24  1.24 

1.16  1.65  1.66 

1.23  1  1  1 

1.56 

1.3 

1.15 

1.46 

1.29 

1.42 

1.21 

scale  output  minimum 
0.77085 

scale  output  maximum 
4.0569 
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Network  1^19 


% 


iq)ut  Bumber  1  20 

dictioinaiy  input  LOG.KDSl  AAF  RELY  DATA  CPLX  TIME  STOR  VIRT  TURN 

ACAP  AEXP  PCAP  VEXP  LEXP  MODP  TOOL  SCED  E  SD  ORG 

ouQni*  number  1  1 

dkdonaiy  output  LOG_MMAC 

hidden  16  19 

filename  train&cts  c:\brain\test\cokealll.fct 
filename  ninfacts  c:\braiD\test\kemmer.iD 
leamrate  0.9000  SO  0.75  7S  0.6000  90  0.5000 
leainlayer  1.0000  1.0000  1.0000 
traintol  0.1000  0.04  0.8000  100 
testtol  0.2000 
random  5.0 
maxruns  304 

fimction  hidden  1  sigmoid  0.0000  1.0000  0.0000  1.00000 

fimction  hidden2  sigmoid  0.0000  1.0000  0.0000  1.00000 

fimction  output  sigmoid  0.0000  1.0000  0.0000  1.00000 

LOG^KDS  AAF  RELY  DATA  CPLX  TIME  STOR  VIRT  TURN 

ACAP  AEXP  PCAP  VEXP  LEXP  MODP  TOOL  SCED  E  SD 

ORG 

LOG^MMA 
scale  input  mmitmiTn 


0.29666  0.43  0.75 
0.95  0.82  0.83 
scale  input  maximuffl 

0.94  0.7  1 

1  000 

1  0.87 

0.87 

0.71 

0.82 

0.7 

0.9 

3.06069  1  1.4 

1.14  1.24  1.24 

scale  output  mminnim 

1.16  1.65  1.66 

1.23  1  1  1 

1.56  1.3 

1.15 

1.46 

1.29 

1.42 

1.21 

0.77085 


scale  ouiput  maxinnun 
4.0569 
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Network  18/15 


ioput  number  1  20 

dictionaiy  input  LOG_KDSI  AAF  RELY  DATA  CPLX  TIME  STOR  VIRT  TURN 

ACAP  AEXP  PCAP  VEXP  LEXP  MODP  TOOL  SCED  E  SD  ORG 

output  number  1  1 

dictionary  output  LOG  MMAC 

hidden  18  IS 

filename  trainfacts  c:\brain\test\cokeaIll.fct 
filename  runfacts  c:\brain\test\keinmer.m 
leamrate  0.9000  SO  0.7S  7S  0.6000  90  O.SOOO 
leamlayer  1.0000  1.0000  1.0000 
traintol  0.1000  0.04  0.8000  100 
tesnol  0.2(K)0 
random  S.O 
maxruns318 

fiinction  hidden  I  sigmoid  0.0000  1.0000  0.0000  1.00000 

fiinction  hidden2  sigmoid  0.0000  1.0000  0.0000  1.00000 

fiinction  output  sigmoid  0.0000  1.0000  0.0000  1.00000 

LOG_KDS  AAF  RELY  DATA  CPLX  TIME  STOR  VIRT  TURN 

ACAP  AEXP  PCAP  VEXP  LEXP  MODP  TOOL  SCED  E  SD 

ORG 

LOG^MMA 
scale  mput  minimum 


0.29666  0.43  0.7S 
0.9S  0.82  0.83 
scale  mput  maximum 

0.94  0.7  1 

1  000 

1  0.87 

0.87 

0.71 

0.82 

0.7 

0.9 

3.06069  1  1.4 

1.14  1.24  1.24 

1.16  1.6S  1.66 

1.23  1  1  1 

1.S6  1.3 

I.IS 

1.46 

1.29 

1.42 

1.21 

scale  output  minimuni 
0.7708S 

scale  output  maximum 
4.0S69 
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r 


Network  24/25 


input  number  1 20 

dictionaiy  input  LX)G_KDSI  AAF  RELY  DATA  CPLX  TIME  STOR  VIRl  TURN 

ACAP  AEXP  PCAP  VEXP  LEXP  MODP  TOOL  SCED  E  SD  ORG 

output  number  1 1 

dictionaiy  output  LOG_MMAC 

bidden  24  25 

filename  trainfiicts  c;\braiD\test\cokeaIll.fct 
filename  runfacts  c:\brain\test\kenimer.in 
leainrate  0.9000  50  0.75  75  0.6000  90  0.5000 
leanlayer  1.0000  1.0000  1.0000 
traintol  0.1000  0.04  0.8000  100 
testtol  0.2000 
random  5.0 
maxruns4i7 

fimction  hidden  1  sigmoid  0.0000  hOOOO  0.0000  1.00000 
fiinction  bidden2  sigmoid  0.0000  1.0000  0.0000  1.00000 
function  output  sigmoid  0.0000  1.0000  0.0000  1.00000 
LOG_KDS  AAF  RELY  DATA  CPLX  TIME  STOR  VIRT  TURN 
ACAP  AEXP  PCAP  VEXP  LEXP  MODP  TOOL  SCED  E  SD 

.-.iX5__MMA 
scale  input  imnmwifn 

0.29666  0.43  0.75  0.94  0.7  1  1  0.87  0.87  0.71  0.82  0  7  0  9 

0.95  0.82  0.83  1  000 

scale  input  maximum 

3.06069  1  1.4  1.16  1.65  1.66  1.56  1.3  1  15  1.46  1.29  142  121 

1.14  1.24  1.24  1.23  1  1  1 

scale  output  "wwiTWum 
0.77085 

scale  output  maximum 
4.0569 
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Brainmakcr  Neural  Network  Training  and  Testing  Files 
COCOMO  Training  and  Kemmerer  Testing  Versions 


COKEALL1.FCT 


Acts 
- 1 

2.05307  1  0.88  1.16  0.7  1  1.06  1.15  1.07  1.19  1.13  1.17  1.1 

1  1.24  1.1  1.04  [  E 

3.30963 
-  2 

2.46686  0.85  0.88  1.16  0.85  1  1.06  1  1.07  1  0.91  1  0.9 

0.95  1.1  1  1  [  E 

3.20412 

- 3 

2.12057  1  1  1,16  0.85  1  1  0.87  0.94  0.86  0.82  0.86  0.9 

0.95  0.91  0.91  1  (  SD 

2.3856 
- 4 

1.77815  0.76  0.75  1.16  0.7  1  1  0.87  1  1.19  0.91  1.42  i 

0.95  1.24  1  1.04  [  ORG 

2.38021 

- 5 

1.20412  1  0.88  0.94  1  1  1  0.87  1  1  1  0.86  0.9 

0.95  1.24  1  1  [  ORG 

1.51851 
- 6 

0.60206  1  0.75  1  0.85  1  1.21  1  1  1.46  1  1.42  0.9 

0.95  1.24  1.1  1  [  ORG 

1.63346 

- 7 

0.83884  1  0.75  1  1  1  1  0.87  0.87  1  1  1  0.9  0.95 

0.91  0.91  1  [  ORG 

0.90309 
- 8 

1.34242  1  1.15  0.94  1.3  1.66  1.56  1.3  1  0.71  0.91  1  1.21 

1.14  1.1  1.1  1.08  [  E 

3.0314 
- 9 
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1.47712  1  1.15  0.94  1.3  1.3  1.21  1.15  1  0.86  1  0.86  1.1 

1.07  0.91  1  1  [  £ 

2.62634 
- 10 

1.46239  0.63  1.4  0.94  1.3  1.11  1.56  1  1.07  0.86  0.82  0.86  0.9 

1  1  1  1  [  E 

2.5065 


1.50S15  0.63  1.4  0.94  1.3  l.Il  1.56  1  1.07  0.86  0.82  0.86  0.9 

1  1  1  1  (  £ 

2.33845 


1.5682  1  1.15  0.94  1.3  1.11  1.06  1  1  0.86  0.82  0.86  1 

0.95  0.91  1  1.08  [  E 

2.30319 
- 13 

1.39794  0.96  1.15  0.94  1.3  1.11  1.06  1.15  1  0.71  1  0.7  1.1 

1  0.82  1  1  [  £ 

1.89762 
- 14 

0.47712  1  1.15  0.94  1.65  1.3  1.56  1.15  1  0.86  1  0.7  1.1 

1.07  1.1  1.24  1.23  [  SD 

1.86332 
- 15 

0.59106  1  1.4  0.94  1.3  1.3  1.06  1.15  0.87  0.86  1.13  0.86  1.21 

1.14  0.91  1  1.23  (  £ 

1.78533 
- 16 

0.78533  0.6  1.4  1  1.3  1.3  1.56  1  0.87  0.86  1  0.86  1 

1  1  1  1  [  E 

1.60206 
- 17 

0.5563  0.53  1.4  1  1.3  1.3  1.56  1  0.87  0.86  0.82  0.86  1 

1  1  1  1  I  £ 

0.95424 
- 18 

2.50515  1  1.15  1.16  1.15  1.3  1.21  1  1.07  0.86  1  1  1 

1  1.24  1.1  1.08  [  £ 

4.0569 
- 19 

3.06069  0.84  1.15  1.08  1  1.11  1.21  0.87  0.94  0.71  0.91  1  1 

1  0.91  0.91  1  [  £ 

3.81954 
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- 20 

2.47567  0.96  1.4  1.08  1.3  1.11  1.21  1.15  1.07  0.71  0.82  1.08 

1.1  1.07  1.24  1  1.08  [  SD 

3.80618 
- —21 

2.4014  1  1  1.16  1.15  1.06  1.14  0.87  0.87  0.86  1  1  1 

1  0.91  0.91  1  [  E 

3.39005 
- 22 

2.07188  0.92  1.15  1  1  1.27  1.06  1  1  0.86  0.82  0.86  0.9 

1  0.91  1  1.23  [  E 

2.85973 
- 23 

1.88649  0.98  1.15  1  1  1.08  1.06  1  1  0.86  0.82  0  86  0  9 

1  1  1  1.23  [  E 

2.73158 
- 24 

1.95424  1  0.88  1  0.85  1.06  1.06  1  0.87  1  1.29  1  1  1 

0.95  0.82  0.83  1  [  SD 

2.65609 
- 25 

1.57978  1  1.15  1.16  1.3  1.15  1.06  1  0.87  0.86  1  0.86  1  1 

1  0.82  0.91  1.08  [  E 

2.7185 
- 26 

1.68124  1  0.94  1  0.85  1.07  1.06  1.15  1.07  0.86  1  0.86  1  1 

1  0.91  1.1  1.08  [  E 

2.58771 
- 27 

0.97312  1  1.15  0.94  1.15  1.35  1.21  1  0.87  1  1  1  1 

1  0.82  1.1  1.08  [  E 

1.94448 
- 28 

1.11394  1  1.15  1.08  1.3  1.11  1.21  1.15  1.07  0.86  1  0.86  1  1 

1.07  1.1  1.1  1  [  ORG 

1.99122 
- 29 

0.33041  1  0.88  1  1  1  1  1  1  1.1  1.29  0.86  1  1 

0.91  0.91  1.23  [  SD 

0.86332 
- 30 

0.29666  1  0.88  1  1  1  1  1  1  1  1.29  0.86  1  1 

0.91  0.91  1.23  [  SD 
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0.77085 
- 31 

1.79239  0.81  1.4  1.08  1  1.48  1.56  1.15  1.07  0.86  0.82  0.86  1.1 

1.07  1  1  1  [  E 

3.02653 
- 32 

2.59106  0.67  0.88  1.08  0.85  1  1  1  1  0.71  0.82  1  1 

1  1.1  1.1  1  [  SD 

2.84633 
- 33 

1.62324  0.96  1.4  1.08  1.3  1.48  1.56  1.15  0.94  0.86  0.82  0.86 

0.9  1  0.91  0.91  1  [  E 

2.78175 
- 34 

1.36172  0.96  1.15  1.08  1  1.06  1  1  0.87  1  1  1  1  1 

0.91  1.1  1.23  [  E 

2.36172 
- 35 

1.11394  1  0.75  0.94  1.3  1.06  1.21  1.15  1  1  0.91  1  1.1 

1  1.24  1.24  1  [  E 

1.91381 
- 36 

1.17609  0.81  0.88  1.08  0.85  1  1  0.87  0.87  1.19  1  1.17  0.9 

0.95  1  0.91  1.04  [  SD 

1.74036 
- 37 

1.77815  0.56  0.88  0.94  0.7  1  1.06  1  1  0.86  0.82  0.86  1 

1  1  1  1  [  ORG 

1.67209 
- 38 

1.17609  1  1  1  1.15  I  1  0.87  0.87  0.71  0.91  1  0.9 

«95  0.82  0.91  1  [  ORG 

*.07918 
- 39 

0.79239  1  1  1  1.15  1  1  0.87  1  0.71  0.82  0.7  1 

0.95  0.91  1.1  1  [  ORG 

0.90309 
- 40 

0.47712  0.83  1  0.94  1.3  1  1  1  0.87  0.86  0.82  1.17  1 

1  1.1  1  1  [  ORG 

0.90309 

- 41 
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0.72427  1  0.88  0.94  1  1  1  0.87  0.87  1  0.82  0.7  0.9 

0.93  0.91  0.91  1  [  ORG 

0.77815 
- 42 

1.65801  0.43  0.88  1.04  1.07  1  1.06  0.87  1.07  0.86  1  0.93  0.9 

0.95  0.95  0.95  1.04  [  ORG 
1.65321 
- 43 

1.45636  0.98  1  1.04  1.07  1  1.21  0.87  1.07  0.86  1  1  0.9 

0.95  1  1  1.04  [  ORG 

1.91907 
- 44 

1.48572  0.98  0.88  1.04  1.07  1.06  1.21  0.87  1.07  1  1  1  0.9 

0.95  1.1  1  1.04  [  ORG 

1.93951 

- 45 

1.54406  0.91  0.88  1.04  1.07  1  1.06  0.87  1.07  1  1  1  0.9 

0.95  1  0.95  1.04  [  ORG 

2.0253 
- 46 

1.86332  0.78  0.88  1.04  1.07  1  1.06  0.87  1.07  1  1  0.86  0.9 

0.95  1  1  1.04  [  ORG 

2.10037 

- 47 

1.36172  1  0.75  0.94  1.3  1  1  0.87  0.87  0.71  0.82  0.7  1.1 

1.07  1.1  1  1.04  [  ORG 

1.5563 
- 48 

2.66651  0.67  0.88  0.94  0.85  1  1  0.87  1  1.19  0.91  1.17  0.9 

0.95  1.1  1  1.04  [  SD 

3.10448 
- 49 

1.95904  1  1  1  0.85  1  1  1  0.87  0.71  1  0.7  1.1  1 

0.82  0.91  1  [  SD 

2.19312 
- 50 

1.38021  1  1.15  1  1  1.3  1.2!  1  0.87  0.86  1  0.86  1.1 

1  1  1  1  [  E 

2.24551 
- 51 

1  1  0.88  1  1  1  1  1  1.15  1.19  1  1.42  1  0.95 

1.24  1.1  1.04  [  ORG 

2.08636 
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0.91381  1  0.88  0.94  0.85  1  1.06  1.15  1  1  1  111 

1.07  1.24  1.1  1  [  ORG 

1.61278 


0.72427  1  0.88  0.94  1.15  1.11  1.21  1.3  1  0.71  1  0.7  1.1 

1.07  1  1.1  1.08  [  SD 

1.14612 


0.64345  1  1  0.94  1  1  1.06  1.15  0.87  1  0.82  1  1 


0.95  0.91  1.1  1  [  ORG 

1.30103 
- 55 


0.79934 

1  0.88  0.94 

0.7 

1 

0.95  1.1 

1.25527 

56 

1  1  [  ORG 

1.43136 

1  1.15  0.94 

1.3 

1.3 

1.07  1.1 

2.98136 

57 

1.1  1.08  [  E 

• 

1.23044  0.87  1  0.94 

1.07  1.1  1.1  1.23  [  E 

2.37474 
- 58 

1.15 

1.11 

1.39794 
0.95  0.91 
2.11394 
- 59 

1  1.4  0.94 

1  1  [  E 

1.3 

1.66 

1.36172 

1  0.91 
1.84509 

60 

0.9  1  0.94 

1  1  [  ORG 

1.15 

1.06 

0.82607 

1  1.15  0.94 

1.3 

1.11 

1.07  1.1 

1.75587 
- 61 

1.1  1.08  (  ORG 

1.44715 

1  0.82 
1.69897 
- 62 

1  1  0.94 

1  1  (  ORG 

1.15 

1 

0.95904 

1  0.88  0.94 

1.3 

1.11 

1.14  0.91  1.24  1  [  SD 

1  0.87  0.87  0.86  0.82  1.17  0.9 

1.21  1  1  0.86  0.91  1  1.1 

1.21  1.3  1  1  1  1  1.1 

1.21  1  1  0.71  0.82  0.7  0.9 

1.06  1  0.87  1  1  1  1 

1.06  1  1  0.86  1.13  0.86  1.1 

1  0.87  0.87  0.86  1  0.86  0.9 

1.21  1.15  1  0.78  0.82  0.7  1.21 
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1.57978 
- 63 

1  1  1  0.94  1.15  1  1  1  0.87  0.71  0.82  0.86  1  1 

0.82  1  1  [  E 

1.17609 


XEMMER.IN 


LOG.KDS  AAF  RELY  DATA  CPLX  TIME  STOR  VIRT  TURN 
ACAP  AEXP  PCAP  VEXP  LEXP  MODP  TOOL  SCED  SD  E 
ORG 
fiictsnm 
- 1 

2.40414  1  1.15  1.08  1.15  1  1.06  0.87  1  0.86  1.13  0.86  1 

1  0.82  0.91  1.02  [  SD 

- 2 


1.60745 

1  1  1 

1 

1 

1  0.87 

1 

1  1.07  0.86  1 

1 

1  1 

1  [  SD 

• 

2.65321 

1  1  1.16 

1 

1.3 

1.21 

0.87 

1 

0.86 

1.13 

1 

1 

1.03  0.91 

A 

1  1  [  E 

2.33122 

1  1  1.08  0.92 

1 

1 

0.87 

0.93 

0.8 

0.89 

0.85 

0.97 

0.95  1 

c 

1  1.02  [  SD 

2.65311 

1  1  0.94 

1 

1 

1  0.93 

0.87 

0.78 

1.13 

0.93 

1.1 

1.03  1 

0.91  1.07  [  ORG 

1.69897 

1  1  1.16 

1 

1.11 

1.06 

1 

0.87 

0.86 

1.13 

0.86 

0.95 

0.97  0.91 
1 

1  1.23  [  E 

1.63346 

1  1  1.16 

1 

1.11 

1.06 

1 

0.87 

0.86 

1.13 

0.7 

0.95 

1.07  0.91 

-  g 

1  1.23  [  E 

2.22271 

1  1.15  1.16 

1 

1 

1 

1 

0.93 

1  0.95 

1  0.95 

0.97  1 

n 

1  1.16  [  SD 

2.46089 

1  1  1.16  1.15 

1.11 

1.06 

0.87  0.87  1 

1.13 

1.06 

1.1 

1  1  1  1.08  [  E 


1.59106  1  1  1.16  1  1.11  1  0.93  1.03  0.71  1  0.7  1.1 

1.14  1.05  1  1.23  [  SD 

2.40517  1  1.16  1  1  1  1  1  1  0.86  1.13  0.86  1  1 

0.86  1  1.23  [  SD 

- 12 

2.10924  1  1  1.16  1  1  1  0.87  0.87  0.86  1.07  0.86  1 

1.03  1.1  0.95  1.08  [  SD 

- 13 

2.2079  1  I  1.16  1  1.06  1  0.87  0.87  1.19  1  0.78  1.21 

0.97  1.1  1  1.08  [  SD 

- 14 

2.21695  1  1  1.16  1  1  1  0.87  0.87  0.86  1.21  0.93  1.1 

1.1  0.95  0.91  1.04  (  ORG 

- 15 

1.77959  1  1  1  1  1  1  1  0.87  0.71  0.91  0.86  1.1 

1.03  0.91  0.91  1  [  SD 
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APPENDIX  F 


Genetic  A^iorithm  Fitnets  Function  (fit.c) 
(Prior  to  instantiation  of  awk  Wrapper) 


4techide  <niath.li> 
struct  entry  { 
doubk  effint; 
double  akdsi; 

int  il42,i3J4,iS,i647,i8,i9,ilO,il  l,il2,il3,il4.ilS; 
intie; 

>; 

struct  entry  tableQ  > 

( 

^include  "table,  c* 

> 


int  size  (sizeof  table)/(sizeoi^  struct  entry)); 


double  f1(y) 
register  double  *y; 

{ 

double  res; 
double  emm; 
double  erq); 
static  double  x[96]; 
roister  int  ij; 


fi»r(i  *  0;  I  <  90;  i++)  { 
xIi]-y(q+200; 
x[il/=  400.0; 
m  *- 1.5; 
xra■^-.5; 

> 
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foi(i-90;i<96;i++){ 
xli]-y[i]  +  200; 
x[il/-IOO; 

) 


fi»i(res  *  0,  i»  0;  i  <  42;  i+«6)  { 
m  (x[il<  xIH- 1)))  res  +-1000; 
ii(!  (x[i]<  x[i+2]))  res  4-1000; 
ii(!  (x[il<  x[i43]))  res  4-1000; 
m  (x(i]<  x[i44]))  res  4-1000; 
iKf  (xr»l<  xri45]))  res  4-1000; 
if(!  (xri4l]<  xli42]))  res  4-1000 
ifl:!  (xri4l]<  x[i43]))  res  4-1000 
ifl[!  (x(i4ll<  x[i+4]))  res  4-1000 
ifl:!(xti4ll<x[i45)))  res  4-1000 
at!  (xti42]<  x[i43]))  res  4-1000 
at!  (x[i42]<  x[i44]))  res  4-1000 
i^!  (xti42]<  x[i45]))  res  4-1000 
at!  (x[i43]<  x[i+4]))  res  4-1000 
at!  (xti43]<  x(i45J))  res  4-1000 
i^!  (xti44]<  x[i45]))  res4-1000; 

) 

foi(i=42;i<84;i4»6)  { 
at!  (x[i]>  x[i4ll))  res  4-1000; 
at!  (x(i]>  x[i42]))  res  4-1000; 
at!  (x[i]>  x[i43]))  res  4-1000; 
at!  (x[i]>  x[i44]))  res  4-1000; 
at!  (x[i]>  xli45]))  res  4-1000; 
at!  (x[i4ll>  x[i42]))  res  4-1000 
at!  (x[i4l]>  x[i431))  res  4-1000 
at!  (xti4l]>  x[i44]))  res  4-1000 
at!  (x[t4l]>  x(i45]))  res  4-1000 
at!  (x[i42]>  x[i43]))  res  4-1000 
at!  (x[i42]>  x[i  4]))  res  4-1000 
at!  (x(i42]>  x[i45i))  res  4-1000 
ifl[!  (x[i43)>  x[i44]))  res  4-1000 
at!  (x[i43]>  x[i45]))  res  4-1000 
ifl:!  (x[i44]>  x[i45]))  res4-1000; 

> 

/*foi<i-90;i<91;i44){ 


Penalty  functions  to  force  GA 
to  obey  rules  established  by 
Boehm  for  the  first  seven  cost 
drivers.  Comment  out  if 
unconstrained  drivers  are 
desired. 


Penalty  functions  to  force  GA 
to  obey  rules  established  by 
Boehm  for  drivers  eight  through 
fourteen.  Comment  out  a 
unconstrained  drivers  are 
desired. 


I 
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ifl;!  (x[il  <  x[i+l]  ))  res  +“1000; 
ifl[!  (xH  <  x[i+2]  ))  res  +-I000; 
i0[!  (x[i+l]  <  xIi+2]))  res  +«1000; 

> 

foi<i^3;i<94;i++){ 
ifl[!  (x[i]  >  x[i+l]  ))  res  +-1000; 
19!  (x[i]  >  xIi+2]  ))  res  +-1000; 
ifl[!  (xIi+1]  >  x(i+2]))  res  +«1000; 

) 

•/ 


Penalty  fimcdons  to  force  GA 
to  obey  rules  establislied  by 
Boehm  for  coefiBdents  and 
exponents.  Comment  out  if 
unconstrained  drivers  are 
desired. 


for(i  *  0;  i  <  size;  i++)  { 

/*  get  product  of  cost  drivers  -  goes  from  0  to  89..  treat  as  a  6x15  array*/ 
emm  *  x{table[i].il]; 
emm  •*  x[table[i].i2  +  6]; 
emm  *•  x(table[i].i3  +  6*2]; 
emm  •*  x[tab]e[i].i4  +  6*3]; 
emm  **  x[table[i].i5  +  6*4]; 
emm  **  x[table[i].i6  +  6*5]; 
emm  **  x[table[i].i7  +  6*6]; 
emm*=x(table[i].i8  +  6*7]; 
emm*-x[UbIe[i].i9  +  6*8]; 
emm  **  x[uble[i].ilO  +  6*9]; 
emm  *=  x[table[i].il  1  +  6*10]; 
emm  *=  x[table[i].il2  +  6*11]; 
emm  **  x[table[i].il3  +  6*12]; 
emm  **  x[tab]e[i].i]4  +  6*13]; 
emm  **  x[table[i].il5  +  6*14]; 
emm  •*  x[table[i].ie+90];  /*  coeff  */ 
emm  *“  pow(table[i].akdsi,  x[table[i].ie+93]);  /*  erq>onent  */ 
res+“ 

8qrt(((table|i].eflbrt-emm)/(tablc|i].efrort))*((tableIi).eflbrt-cmm)/(table(i].efTort))); 

) 

return  res/size; 

> 

/*GAevain  I0;200dg96*/ 


Bold  underlined  type  is  GAucsd  1.4  declaration 
for  the  diromosome.  Translate  as  10  bits  per 
nuniber,  range  *200  to  200, 96  numbers  total 


Bold  type  is  fitness  measure 
evaluation.  Average  relative  error  is 
drown. 
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Genetic  AlsoriChni  Input  Data  File:  COCOMO  Training  and  Testing  Version 


Format  is  {MM^ActuaU  Adjusted  KDSl,  15  Cost  Drivers,  Mode 

literal  Driver  Values  Defined  as: 

0*  Veiy  Low 
l*Low 
2  »  Nominal 
3-Hi^ 

4*  Verylfigb 
5  •Extralfigli 

Mode  Defined  as: 

0  *  Embedded 
1 »  Senudetached 
2  »  Organic 


•’Table.c’ 


(2040,  113,  1,  4,  0,  2,  3,  3,  3,  1,  1,  1,  1,  2,  0,  1.  3,  0), 
(243,  132,  2,  4,  1,  2,  2,  1,  1,  3,  4,  3,  3,  3,  3,  3,  2,  1). 
<240,  43.6,  0,  4,  0,  2,  2,  1,  2,  1,  3,  0,  2,  3,  0,  2,  3,  2), 

(33,  16,  1,  1,  2,  2,  2,  1,  2,  2,  2,  3,  3,  3.  0,  2,  2,  2), 

(423,  30,  3,  1,  4,  4,  4,  3,  2,  3,  2,  3,  1,  1,  3,  2,  2,  0), 

(321,  18.27,  4,  1,  4.  3,  5,  2,  3,  3,  4,  3,  3,  2,  2,  2,  2,  0). 

(218,  20.16,  4,  1,  4,  3,  5,  2,  3,  3,  4,  3,  3,  2,  2,  2,  2,  0}, 

(40,  3.66,  4,  2,  4,  4,  5,  2,  1,  3,  2,  3,  2,  2,  2.  2,  2,  0), 
(6400,  287.04,  4,  3,  4,  3,  4,  3,  3,  4,  4,  2,  1,  1,  0,  2,  1,  1), 

(2455,  252,  2,  4,  3,  3,  3,  1,  1,  3,  2,  2,  2,  2,  3,  3,  2,  0), 

(724,  108.56,  3,  2,  2,  4,  3,  2,  2,  3,  4,  3,  3,  2,  3,  2,  0,  0), 
(539,  75.46,  3,  2,  2,  3,  3,  2,  2,  3,  4,  3,  3,  2,  2,  2,  0,  0), 

{453,  90,  1,  2,  1,  3,  3,  2,  1,  2,  0,  2,  1,  3,  4,  4,  2,  1), 

(523,  38,  3,  4,  4,  3,  3,  2,  1,  3,  2,  3,  1,  2,  4,  3,  1,  0), 

(387,  48,  1,  2,  1,  3,  3,  3,  3,  3,  2,  3,  1,  2,  3,  1,  1,  0), 

(98,  13,  3,  3,  4,  3,  4,  3,  3,  3,  2,  3,  1,  1,  1,  1,  2,  2), 

(1063,  50.22,  4,  3,  2,  4,  5,  3,  3,  3,  4,  3,  1,  1,  2,  2,  2,  0}, 
(702,  261.3,  1,  3,  1,  2,  2,  2,  2,  4,  4,  2,  2,  2,  1,  1,  2,  1), 

(605,  40.32,  4,  3,  4,  4,  5,  3,  1,  3,  4,  3,  3,  2,  3,  3,  2,  0}, 
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{230,  22.08,  3,  3,  2,  3,  2,  2,  1,  2,  2,  2,  2,  2.  3.  1,  0,  0}, 
(82,  13,  0,  1,  4,  3,  4,  3,  2,  2,  3,  2,  1,  2,  0,  0,  2,  0), 

(55,  12.15,  1,  3,  1,  2,  2,  1,  1,  1,  2,  1,  3,  3,  2,  3,  3,  1), 

(8,  2.49,  2,  1,  4,  2,  2,  2,  1,  3.  4,  1,  2,  2,  1,  2,  2.  2}, 

(6,  5.3,  1,  1,  2,  2,  2,  1,  1,  2,  4,  4.  3,  3,  3,  3,  2,  2}, 

(45,  19.565,  1,  2,  2,  2,  3,  1,  3,  3,  2,  3,  3,  3,  3,  3,  3,  2}, 

(83,  28.028,  2,  2,  2,  2,  4,  1,  3,  3,  2,  2,  3,  3,  2,  2,  3,  2), 

(87,  29.988,  1,  2,  2,  3,  4,  1,  3,  2,  2,  2,  3,  3,  1,  2,  3,  2}, 

(106,  31.85,  1.  2,  2,  2,  3,  1,  3,  2,  2,  2,  3,  3,  2.  3.  3,  2}, 

(126,  56.94,  1,  2,  2,  2,  3,  1,  3,  2,  2,  3.  3.  3,  2,  2,  3,  2}, 

(36,  23,  0,  1,  4,  2,  2,  1,  1,  4,  4,  4,  1,  1,  1,  2,  3,  2}, 

(1272,  310.88,  1,  1,  1,  2,  2,  1,  2,  1,  3,  1.  3,  3.  1,  2,  3,  1}, 

(156,  91,  2,  2,  1,  2,  2,  2,  1,  4,  2,  4,  1,  2.  4.  3,  2,  1), 

(176,  24,  3,  2,  2,  4,  4,  2,  1,  3,  2,  3.  1,  2,  2,  2,  2,  0}, 

(122,  10,  1,  2,  2,  2,  2,  2,  4,  1,  2.  0.  2,  3,  0.  1,  3,  2}. 

(14,  5.3,  1,  1.  3,  3,  4,  4,  2,  4,  2,  4,  1,  1,  2,  1,  1,  1}. 

(20,  4.4,  2,  1,  2,  2,  3,  3,  1,  2,  4,  2,  2,  3,  3,  1,  2,  2}, 

(18,  6.3,  1,  1,  0,  2,  2,  1,  1,  3,  4,  1,  3,  3,  1,  2,  2,  2), 

(958,  27,  3,  1,  4,  4,  4,  2,  2,  3,.  3,  2,  1,  1,  1,  1,  1,  0), 

(237,  14.79,  2,  1.  3,  3,  4,  4,  2,  2,  2,  2,  1,  1,  1,  1,  0,  0}, 

(130,  25,  4,  1,  4,  5,  4,  2,  2,  4,  4,  4,  3,  3,  3,  2,  2,  0}, 

(38,  9.1,  1,  1,  4,  3,  4,  3,  2,  4,  4,  4,  0.  0,  3,  0,  2,  1), 

(15,  10,  2,  1,  3,  2,  2,  2,  1,  4,  4,  3,  2,  2,  4,  2,  2,  0), 
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APPENDKG 


Genetic  Programming  Fitness  Function  File 
*Titncss.c** 


^include  <math.h> 

/• 

SGPC;  Sinople  Genetic  Programnmg  in  C 

(c)  1993  by  Waher  Alden  Tackett  and  Aviram  Canni 

This  code  and  documentation  is  copyrighted  and  is  not  in  the  public  domain. 

AH  rights  reserved. 

.  This  notice  may  not  be  removed  or  ahered. 

•  You  may  not  try  to  make  money  by  distributing  the  package  or  by  usmg  the 
process  that  the  code  creates. 

•  You  may  not  distribute  modified  versions  without  clearly  documentmg  your 
dianges  and  notifying  the  princ^al  author. 

•  The  origin  of  this  software  must  not  be  misrepresented,  either  by 
e)q)licit  claim  or  by  omission.  Since  few  users  ever  read  sources, 
credits  must  appear  in  the  documentation. 

•  Ahered  versions  must  be  plainly  marked  as  such,  and  must  not  be 
misrq[>resented  as  being  the  original  software.  Since  few  users  ever  read 
sources,  CTedhs  must  appear  in  the  documentation. 

•  The  authors  are  not  re^onsible  for  the  consequences  of  use  of  this 
software,  no  matter  how  awful,  even  if  they  arise  fi-om  flaws  in  it. 

If  you  make  changes  to  the  code,  or  have  suggestions  for  changes, 

let  us  know:  (^c@ipld01.hac.com) 

•/ 

^fifiideflint 
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lUrtic  diar  fitness_c_rcadQ*"$Id;  fitness.c,v  2.8  1993/04/14  05:19:57  g>c-avc  Eiq) 

gpc>avc  $"; 

ie&dif 

/• 

* 

*  SLog:  fitness.c,v  $ 

*  Revision  2.8  1993/04/14  05:19:57  gpoavc 

*  jua  a  revisioD  number  change 

•/ 

delude  <adio.h> 
include  <maIloc.h> 

#include  <enno.h> 

Ifyxdbxit  ''s>c.h'' 

#mchide  EXP 
«inchideVAL_£XP 

extern  termmal_table_entiy  tennmal_tableQ; 
extern  int  teniiinal_table_sz ; 
extern  int  tota]_vars ; 
extern  int  fitness_table  sz; 

/•  •/  ” 
tl«fdefANSI_FUNC 
VOID  defiae_fitness_case8( 
hit  numpops, 

hit  numgens, 

pop_struct  *pop 

) 

#else 

VOID  define_fitness_cases(n\mq)ops,numgens,pop) 
hit  numpops; 

hit  numgens; 

pq;>_8truct  *pop; 

#endif 

{ 

#if0 

/*  this  tengilate  nukes  all  of  the  fitness  cases  the  same  for  all 
populations,  but  that  isn't  necessary  */ 
hitp4; 
float  range; 

Bumfc  »  10;  /*  nuniber  of  fitness  or  training  cases  */ 
numte  «  10;  /*  number  of  tea  or  validation  cases  */ 
range  » 1.0; 
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ftness_c«sesjuble  «  (float  **)  nia0oc(mm^ops*azeo^;floit  •)); 
test_caMS_table  *  (float  **)  inalloc(nunipops*B2eof(float  •)); 
fitnessjMsesjtablcjout  *  (float  •*)  inaDoc(flun5>ops*azeofl[float  *)); 
test_cases_table  out  •  (float  **)  inafloo(nuxnpops*a2*o^float  *)); 
for  (p-0;  p<Dunpops;  { 

cases_table(p]  *  (float  *)  iDalloc(nunifc  *  sizeoflfloat)); 
t5st_cascsjtable[p]  *  (float  •)  iiianoc(nuxDtc  *  ^oflfloat)); 
fitoess_casesjUiblejout[p]  *  (float  •)  inaI]oc(DuiQfc  *  Meoflfloat)); 

cases  table__out[p]  *  (float  *)  iDaDoc(iiuintc  •  dacoflfloat)); 
for  (W);  i*^u]iifc;  rH-)  { 

/*  evenly  ipac^  zero..Vange';  */ 
fitnesr_casesjtable|p](i]  •  range  •  (floaty  /  (float)nuinfc; 

/*  evenly  spaced  Yange'  about  zero: 

filness_cases_tablelplti]  -  (2.0*range*(float)i/(flo«)o«®ft)-range; 

V 

fitness_cases_table_outIp][i]  * 
regresaon  fonction(fitness  cases  table[p][i]); 

} 

for  (i*0;  i<nuiittc;  i++)  { 

/*  validation  test  cases  same  as  the  training  set  */ 
test_cases_Ubletp3[i]  *  fitness_cases_tablelp]li); 

/*  validation  could  be  random  within  the  range: 
testjcasesjtable(p][i]  ®  random_float(range); 
or  range  centered  about  zero: 

test  cases_table[p][i]  *  random_float(2.0*range)“range; 

test  cases  table  out[p][i]  *  regresaonjfunction(test_cases_tableIp][i]); 
}  '  "  “ 

> 

ti^endif 

/*  no  processing  needed  —  we've  got  data!  */ 

> 

#ifdefANSI„FUNC 
VOID  evaluate_fitness_of_popuUtions( 
ant  numpops, 

hit  numgens, 

popjBtTUCt*pop, 
hit  p 

) 

#dse 

VOIDevaluate_fimess_ofjpopulations(nuirpops^umgens,pop,p) 
hit  numpt^s; 

hit  numgens; 
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po|>_itnict*pop; 
int  p; 

«aidif 

{ 

Bt  4 

fiv  (H);  r^op[p].pq)ii]atioD_size;  fH-)  { 
popIp3.st«n^rdi^_fitness[i] « 
evalutte  fitness  oHDdividua](pop,p,pop[p].pq>ulsti(»[Q,i); 

> 

> 

#ifdefANSI_FUNC 
float  eva]uate_fitness_oHndividual( 
pop_stnict*pop, 
int  p, 

tree  *4 

int  i 

) 

#dse 

float  eva]uate_fitness_o^individua](pop,  p,  t,  i) 
pop_stnict*pop; 


mt 

tree 

p; 

t; 

Bold  type  defines  the  fitness 

int 

i; 

#endif 

mctsare  of  testing  data  set. 

{ 

Average  relative  error  is  diown 

int 

j; 

float 

res,sum  =  0.0; 

fiw  (H);  j<fitne8sjlable_8z;  j++)  { 
load_tenninaI_8et_vaIues(pop,p,&(fitnessjtable_t[j][l])); 
sum  (float)sqrt((donbie)(fitncss.table.t|jI|0]»eval(t))* 

(fitness  table  tljl{0]-cvai(t))/ fitness  table  t[j]|0] /fitness  table_t[jl(0]); 

res  K  snin/((GENERIC)fitness_table_sz); 
ii(faII_vars(pop,p,t))  { 
res  *=100; 

) 

retun  res; 

} 

#ifiIefANSI_FUNC 
float  validate_fitness__o^tree( 
nt  nunq>ops, 

int  numgens, 

pop_stnict  *pop. 
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Bt  p, 

tree  *1 

) 

float  va]idateJbness_oOfee(nunipops,  numgens,  pop,  p,  t) 
int  nunpops; 

Bt  auiqgeas; 

pop_struct  *pop; 
nt  p; 

tree  •t; 

#cadif 

( 

Bt  j; 

float  res,  sum  0.0; 

float  ev; 

extern  int  val_fitoess_table_sz; 
ft>r(j=0;j<val_fitness_table_sz;j++)  { 
load_teniiinaI_set_yalues(pop,p,&(val_fitsess_tab]e_t[j][l])); 
ev  -  evaKt); 

sum  +■  (float)8qrt((double)(val_fitnets_table_tlj|10j-ev)* 
(val_fitnefs_tabie_t(j](0]-ev)  /  val_fitDess_table_tjj]|0]  / 
val  fitness  table  t(j](0]); 

)- 

res  *  sum/((GENEIUC)val_fitness_table_sz); 
if(!alI_vars(pop,p,t))  { 
res  *=  100; 

} 

return  res; 

} 


#ifdefANSI_FUNC 
int  tenninate_early( 
int  nunq)ops, 

int  numgens, 

pop  struct  *pop 

) 

iMse 

mt  temnnate_eariy(nunq)ops4iumgens,pop) 
int  nunpops; 

int  numgens; 

pop_8truct  *pop; 

#aidif 


Bold  type  defines  the  fitness  measure  of 
testing  data  set.  Average  relative  error 
is  shown. 
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{ 

intpA 

for  (p^;  p<nuiq)<^s;  p^-f)  { 
for  (W);  h^op^].populatioD_aze;  i++)  { 
if  0)q)[p].stindardi»dJ5!ness(i]  <*»  0.0)  { 
return  1; 

> 

} 

> 

retuin  0; 

) 


Genetic  Programming  Training  Data  Input  File 
COCOMO  Training  and  Testing  Version 
•’cotr42.c” 


#include  <stdio.li> 

^include  "s)c.h" 
delude  ‘‘prob.h* 

^include  "cotrdl.!!* 

GENERIC  randomjconstantO; 
tenninaIjUble_entTy  tenmnaljtableQ  =  { 
{0,  ''XO”jandom_constant}, 

{1,  ”Xr^dom_constant}, 

(2,  ”X2”,nndom_canstant}, 

{3,  "X3''^dom_constant}, 

(4,  *X4”^dom_constant}, 

{5,  "X5''4«ndom_constant}, 

{6,  ”X6”^dom_constant}, 

{7,  ”X7”,random_constant}, 

{8,  ”X8”^dom_c<mstant}, 

{9,  ”X9”^dom_constant}, 

{10,  *X10''^dom_constant}, 

(11,  ”Xir^dom_constant}, 

(12,  '^12"^dom_constant}, 

{13,  ”X13”,random_ccMistant}, 

{14,  *X14”,random_constant}, 

{ 1 5,  ”X  15'‘,random_constant}, 


X0»KDSI 

X1=AAF 

X2  through  X16  are  the 
cost  driver  values,  RELY 
throu^  SCED. 
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{16,  ”X16”,rfndom_coiisuuit}, 
{0,FORMATjandom_coiisuuit} 


>; 


at  tenniiudjtable_sz*  sizeofl[temimaljUtbleysizeo9tenDma]juble_eotiy)  •  1; 
at  toUl_vars  *  FITNESS_HEIGHT ; 

GENERIC  fitneMjublej[FriNESS_WroTO3[FITNESS_HEIGHTl  -  {  (2040.0000, 


113.0000, 

1.0000, 

0.8800, 

1.1600, 

0.7000, 

1.0000, 

1.0600, 

1.1500, 

1.0700, 

1.1900, 

1.1300, 

1.1700, 

1.1000, 

1.0000, 

1.2400, 

1.1000, 

1.0400, 

). 

(243.0000,  132.0000,  1.0000, 

1.0000, 

1.1600, 

0.8500, 

1.0000, 

1.0000, 

0.8700, 

0.9400, 

0.8600, 

0.8200, 

0.8600, 

0.9000, 

0.9500, 

0.9100, 

0.9100, 

1.0000, 

(240.0000,  60.0000,  0.7600, 

1 

0.7500, 

1.1600. 

0.7000, 

1.0000, 

1.0000, 

0.8700, 

1.0000, 

1.1900, 

0.9100, 

1.4200, 

1.0000, 

0.9500, 

1.2400, 

1.0000, 

1.0400, 

}. 

{ 33.0000,  16.0000,  1.0000, 

1 

0.8800, 

0.9400, 

1.0000, 

1.0000, 

1.0000, 

0.8700, 

1.0000, 

1.0000, 

1.0000, 

0.8600, 

0.9000, 

0.9500, 

1.2400, 

1.0000, 

1.0000, 

(423.0000,  30.0000,  1.0000, 

1.1500,  1 

0.9400, 

1.3000, 

1.3000, 

1.2100, 

1.1500, 

1.0000, 

0.8600, 

1.0000, 

0.8600, 

1.1000, 

1.0700, 

0.9100, 

1.0000, 

1.0000, 

(321.0000,  29.0000,  0.6300, 

1.4000, 

0.9400, 

1.3000, 

1.1100, 

1.5600, 

1.0000, 

1.0700, 

0.8600, 

0.8200, 

0.8600, 

0.9000, 

1.0000, 

1.0000, 

1.0000, 

1.0000, 

(218.0000,  32.0000,  0.6300, 

1.4000, 

0.9400, 

1.3000, 

1.1100, 

1.5600, 

1.0000, 

1.0700, 

0.8600, 

0.8200, 

0.8600, 

0.9000, 

1.0000, 

1.0000, 

1.0000, 

1.0000, 

(40.0000,  6.1000 

0.6000, 

1.4000, 

1.0000, 

1.3000, 

1.3000, 

1.5600, 

1.0000, 

0.8700, 

0.8600, 

1.0000, 

0.8600, 

1.0000, 

1.0000, 

1.0000, 

1.0000, 

1.0000, 

}, 

(6400.1 

0000,  299.0000,  0.9600, 

1.4000, 

1.0800, 

1.3000, 

1.1100, 

1.2100, 

1.1500, 

1.0700, 

0.7100, 

0.8200, 

1.0800, 

1.1000, 

1.0700, 

1.2400, 

1.0000, 

1.0800,  ), 
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{2455.0000,  252.0000,  1.0000.  1.0000,  1.1600, 

1.1500,  1.0600,  1.1400,  0.8700,  0.8700,  0.8600, 

1.0000,  1.0000,  1.0000,  1.0000,  0.9100,  0.9100, 

1.0000,  }, 

(724.0000,  118.0000,  0.9200,  1.1500,  1.0000, 

1.0000,  1.2700,  1.0600,  1.0000,  1.0000,  0.8600, 

0.8200,  0.8600,  0.9000,  1.0000,  0.9100,  1.0000, 

1.2300,  ), 

(539.0000,  77.0000,  0.9800,  1.1500,  1.0000, 

1.0000,  1.0800,  1.0600,  1.0000,  1.0000,  0.8600, 

0.8200,  0.8600,  0.9000,  1.0000,  1.0000,  1.0000, 

1.2300,  >, 

(453.0000,  90.0000,  1.0000,  0.8800,  1.0000, 

0.8500,  1.0600,  1.0600,  1.0000,  0.8700,  1.0000, 

1.2900,  1.0000,  1.1000,  0.9500,  0.8200,  0.8300, 

1 0000  } 

(523.0000’  38.0000,  1.0000,  1.1500,  1.1600, 

1.3000,  1.1500,  1.0600,  1.0000,  0.8700,  0.8600, 

1.0000,  0.8600,  1.1000,  1.0000,  0.8200,  0.9100, 

1.0800,  }, 

(387.0000,  48.0000,  1.0000,  0.9400,  1.0000, 

0.8500,  1.0700,  1.0600,  1.1500,  1.0700,  0.8600, 

1.0000,  0.8600,  1.1000,  1.0000,  0.9100,  1.1000, 

1.0800,  ), 

(98.0000,  13.0000,  1.0000,  1.1500,  1.0800, 

1.3000,  1.1100,  1.2100,  1.1500,  1.0700,  0.8600, 

1.0000,  0.8600,  1.1000,  1.0700,  1.1000,  1.1000, 

1 0000  } 

{1063.0000,  62.0000,  0.8100,  1.4000,  1.0800, 

1.0000,  1.4800,  1.5600,  1  1500,  1.0700,  0.8600, 

0.8200,  0.8600,  1.1000,  1.0700,  1.0000,  1.0000, 

1.0000,  ), 

(702.0000,  390.0000,  0.6700,  0.8800,  1.0800, 

0.8500,  1.0000,  1.0000,  1.0000,  1.0000,  0.7100, 

0.8200,  1.0000,  1.0000,  1.0000,  1,1000,  1.1000, 

1 0000  } 

(605.0000’,  42.0000,  0.9600,  1.4000,  1.0800, 

1.3000,  1.4800,  1.5600,  1.1500,  0.9400,  0.8600, 

0.8200,  0.8600,  0.9000,  1.0000,  0.9100,  0.9100, 

1 0000  } 

(230.0000,  23.0000,  0.9600,  1.1500,  1.0800, 

1.0000,  1.0600,  1.0000,  1.0000,  0.8700,  1.0000, 
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1.0000, 


1.0000, 


0.9100, 


1.1000, 


1.0000.  1.0000, 

1.2300,  >, 

{ 82.0000,  13.0000,  1.0000,  0.7500,  0.9400, 

1.3000,  1.0600,  1.2100,  1.1500,  1.0000,  1.0000, 

0.9100,  1.0000,  1.1000,  1.0000,  1.2400,  1.2400, 

1.0000,  }, 

{ 55.0000,’  15.0000,  0.8100,  0.8800,  1.0800, 

0.8500,  1.0000,  1.0000.  0.8700,  0.8700,  1.1900, 

1.0000,  1.1700,  0.9000,  0.9500,  1.0000,  0.9100, 

1.0400,  }, 

{  8.0000,’  3.0000,  0.8300,  1.0000,  0.9400, 

1.3000,  1.0000,  1.0000,  1.0000,  0.8700,  0.8600, 

0.8200,  1.1700,  1.0000,  1.0000,  1.1000,  1.0000, 

1.0000,  }, 

{  6.0000,  5.3000,  1.0000,  0.8800,  0.9400, 

1.0000,  1.0000,  1.0000,  0.8700,  0.8700,  1.0000, 

0.8200,  0.7000,  0.9000,  0.9500,  0.9100,  0.9100, 

1.0000,  ), 

{ 45.0000,  45.5000,  0.4300,  0.8800,  1.0400, 

1.0700,  1.0000,  1.0600,  0.8700,  1.0700,  0.8600, 

1.0000,  0.9300,  0.9000,  0.9500,  0.9500,  0.9500, 

1.0400,  }, 

{ 83.0000,  28.6000,  0.9800,  1.0000,  1.0400, 

1.0700,  1.0000,  1.2100,  0.8700,  1.0700,  0.8600, 

1.0000,  1.0000,  0.9000,  0.9500,  1.0000,  1.0000, 

1.0400,  }, 

{87.0000,  30.6000,  0.9800,  0.8800,  1.0400, 

1.0700,  1.0600,  1.2100,  0.8700,  1.0700,  1.0000, 

1.0000,  1.0000,  0.9000,  0.9500,  1.1000,  1.0000, 

1.0400,  }, 

{106.0000,  35.0000,  0.9100,  0.8800,  1.0400, 

1.0700,  1.0000,  1.0600,  0.8700,  1.0700,  1.0000, 

1.0000,  1.0000,  0.9000,  0.9500,  1.0000,  0.9500, 

1.0400,  }, 

{126.0000,  73.0000,  0.7800,  0.8800,  1.0400, 

1.0700,  1.0000,  1.0600,  0.8700,  1.0700,  1.0000, 

1.0000,  0.8600,  0.9000,  0.9500,  1.0000,  1.0000, 

1.0400,  ), 

{ 36.0000,  23.0000,  1.0000,  0.7500,  0.9400, 

1.3000,  1.0000,  1.0000,  0.8700,  0.8700,  0.7100, 

0.8200,  0.7000,  1.1000,  1.0700,  1.1000,  1.0000, 

1.0400,  ), 
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0.6700, 


0.8800, 


0.9400, 


{1272.0000,  464.0000, 

0.8300,  1.0000,  1.0000,  0.8700,  1.0000,  1.1900, 

0.9100,  1.1700,  0.9000,  0.9500,  1.1000,  1.0000, 

1.0400,  ). 

(156.0000,  91.0000,  1.0000,  1.0000,  1.0000, 

0.8500,  1.0000,  1.0000,  1.0000,  0.8700,  0.7100, 

1.0000,  0.7000,  1.1000,  1.0000,  0.8200,  0.9100, 

1.0000,  ), 

(176.0000,  24.0000,  1.0000,  1.1300,  1.0000, 

1.0000,  1.3000,  1.2100,  1.0000,  0.8700,  0.8600, 

1.0000,  0.8600,  1.1000,  1.0000,  1.0000,  1.0000, 

1.0000,  }, 

(122.0000,  10.0000,  1.0000,  0.8800,  1.0000, 

1.0000,  1.0000,  1.0000,  1.0000,  1.1500,  1.1900, 

1.0000,  1.4200,  1.0000,  0.9500,  1.2400,  1.1000, 

10400,  }, 

(  14.0000,  5.3000,  1.0000,  0.8800,  0.9400, 

1.1500,  l.nOO,  1.2100,  1.3000,  1.0000,  0.7100, 

1.0000,  0.7000,  1.1000,  1.0700,  1.0000,  1.1000, 

1.0800,  }, 

( 20.0000,  4.4000,  1.0000,  1.0000,  0.9400, 

1.0000,  1.0000,  1.0600,  1.1500,  0.8700,  1.0000, 

0.8200,  1.0000,  1.0000,  0.9500,  0.9100,  1.1000, 

1.0000,  }, 

(  18.0000,  6.3000,  1.0000,  0.8800,  0.9400, 

0.7000,  1.0000,  1.0000,  0.8700,  0.8700,  0.8600, 

0.8200,  1.1700,  0.9000,  0.9500,  1.1000,  1.0000 

1.0000,  ), 

(958.0000,  27.0000,  1.0000,  1.1500,  0.9400, 

1.3000,  1.3000,  1.2100,  1.0000,  1.0000,  0.8600, 

0.9100,  1.0000,  l.IOOO,  1.0700,  l.IOOO,  1  1000 

1.0800,  }, 

(237.0000,  17.0000,  0.8700,  1.0000,  0.9400, 

1.1500,  1.1100,  1.2100,  1.3000,  1.0000,  1.0000, 

1.0000,  1.0000,  1.1000,  1.0700,  1.1000,  1.1000, 

1.2300,  ), 

(130.0000,  25.0000,  1.0000,  1.4000,  0.9400, 

1.3000,  1.6600,  1.2100,  1.0000,  1.0000,  0.7100, 

0.8200,  0.7000,  0.9000,  0.9500,  0.9100,  1 0000 

1.0000,  }, 

(38.0000,  9.1000,  1.0000,  0.8800,  0.9400, 

1.3000,  1.1100,  1.2100,  1.1500,  1.0000,  0.7800, 
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1.2100, 


1.1400, 


0.9100, 


1.2400, 


0.8200,  0.7000, 

1.0000,  }, 

{  15.0000,  10.0000,  1.0000,  1.0000,  0.9400, 

1.1500,  1.0000,  1.0000,  1.0000,  0.8700,  0.7100, 

0.8200,  0.8600,  1.0000,  1.0000,  0.8200,  1.0000, 

1.0000,  }, 

); 

int  fitness_table_iz  »  FITNESS  WIDTH; 

C^NERIC  fitoess_Uble[FrrNESS_HEIGHT)[Fm4ESSJVIDTH); 


Perl  Script  for  Creating  Genetic  Programming  Input  Training  Data  File 

'*denne.pr* 

(Author:  Ranjan  Bagchi) 


#!4>ublic/gDu/btD/perl  - 

* 

#  define  —  generates  static  data  instantiating  all  necessary 
U  terminals. 

$first»0; 

Soutf  *  SARGVIO]; 
open(OUT,">$outfc"); 
open(OUT_h,">Sout£h"); 
v^e(o)  { 
il(nw*$/)  { 
next; 

} 

@I  =  spftC'); 

illSfirst »  0)  { 
print  OUT 

''#mclude  <stdio.h>\n'', 

•itinclude\''gpc.h\"\n", 

•#mclude\"prob.b\"\n", 
q>rintfi[”#indude  \"%s.h\”\n”,$outf), 

*^G£NERIC  random_cottstant();4" 


print  OUT 

'terminal  tablejentrytermmal  tableQ^  {Vn”; 
foi(0..$#l.l)'( 


126 


prind(OUT  -\t{%d,  \-X%d\“^dom_constant}.\n".$_,$J; 

) 

print  OUT  ”\t(0,FORMAT,nmdom_constant},\n'‘; 
print  OUT  "};\n\ 

int  temiinaljuble_sz  *  Bzeo^tenninaljUble)/sizeo^tenninaljtable_entry)  *  l;\n'‘; 
printf  (OUT  "int  total  vars  -  FrrNESS_HEIGHT;\n"); 

printf (OUT  "GENEWC  fitness  table_t[FITl^SSJVIDTH][FITNESS_HEIC»rr|  - 

n 

Sfonnat  -  •U("."%8.4f;\t"x($#l+l).-},\n"; 

) 

$first++; 

printfi;OUT  Stbnnat,@l); 


> 

printl(OUT  "};\n"); 

print  OUT  "int  fitness_table_sz  =  FITNESS_WIDTH;\n"; 

print  OUT  "GENERIC  fitness_uble[nTNESS_HEIGHTl[FrrNESS_WIDTH];"; 

close(OUT); 

printHOUTJ  "#define  FITNESS  HEIGHT  %d\D".  $«+l); 
printHOUT  h  "#define  FITNESS  WIDTH  %d\n",  Sfirst); 
print!(OUT’ h  "\n\nexteni  GENERIC 
fitness  table[FTTNESS  HEIGHTl[FITNESS_WIDTH];\n-); 
print^OUT^h  "\n\nexteni  GENERIC 
fitness_table_t[FITNESS_WIDTH][FITNESS_HEIGHT];\n"); 

close  (OUTJi); 


Genetic  Programming  input  Testing  Data  File 
COCOMO  Training  and  Testing  Version 
•'cote21.c*’ 


irinchide  <stdio.b> 
trindude  "gpc.h" 
#include  "prob.h" 
^include  "val_cote21.h" 
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CSNERICval  fiiness_ttble  t[VAL  FrrNESS_WIDTH][VAL_FITNESS_HEIGHT]  =  { 
{1600.0000,  293.0000,  0.8500,  0.8800,  1.1600,  0.8500, 

1.0000,  1.0600,  1.0000,  1.0700,  1.0000,  0.9100, 

1.0000,  0.9000,  0.9500,  1.1000,  1.0000,  1.0000, 

). 

{ 43.0000,  4.0000,  1.0000,  0.7500,  1.0000, 

0.8500,  1.0000,  1.2100,  1.0000,  1.0000,  1.4600, 

1.0000,  1.4200,  0.9000,  0.9500,  1.2400,  1.1000, 

10000,  ), 

{  8.0000,  6.9000,  1.0000,  0.7500,  1.0000, 

1.0000,  1.0000,  1.0000,  0.8700,  0.8700,  1.0000, 

1.0000,  1.0000,  0.9000,  0.9500,  0.9100,  0.9100, 

1.0000,  ), 

{1075.0000,  22.0000,  1.0000,  1.1500,  0.9400, 

1.3000,  1.6600,  1.5600,  1.3000,  1.0000,  0.7100, 

0.9100,  1.0000,  1.2100,  1.1400,  1.1000,  1.1000, 

1.0800,  }, 

{201.0000,  37.0000,  1.0000,  1. 1500,  0.9400, 

1.3000,  1.1100,  1.0600,  1.0000,  1.0000,  0.8600, 

0.8200,  0.8600,  1.0000,  0.9500,  0.9100,  1.0000, 

1.0800,  }, 

{ 79.0000,  25.0000,  0.9600,  1.1500,  0.9400, 

1.3000,  1.1100,  1.0600,  1.1500,  1.0000,  0.7100, 

1.0000,  0.7000,  1.1000,  1.0000,  0.8200,  1.0000, 

1.0000,  }, 

{73.0000,  3.0000,  1.0000,  1.1500,  0.9400, 

1.6500,  1.3000,  1.5600,  1.1500,  1.0000,  0.8600, 

1.0000,  0.7000,  1.1000,  1.0700,  1.1000,  1.2400, 

1.2300,  }, 

{ 61.0000,  3.9000,  1.0000,  1.4000,  0.9400, 

1.3000,  1.3000,  1.0600,  1.1500,  0.8700,  0.8600, 

1.1300,  0.8600,  1.2100,  1.1400,  0.9100,  1.0000, 

1.2300,  ), 

{  9.0000,  3.6000,  0.5300,  1.4000,  1.0000, 

1.3000,  1.3000,  1.5600,  1.0000,  0.8700,  0.8600, 

0.8200,  0.8600,  1.0000,  1.0000,  1.0000,  1.0000, 

1.0000,  >, 

{11400.0000,  320.0000,  1.0000,  1.1500,  1.1600, 

1.1500,  1.3000,  1.2100,  1.0000,  1.0700,  0.8600, 

1.0000,  1.0000,  1.0000,  1.0000,  1.2400,  1.1000, 

1.0800,  }, 

{6600.0000,  1150.0000,  0.8400,  1.1500,  1.0800, 

1.0000,  1.1100,  1.2100,  0.8700,  0.9400,  0.7100, 
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0.9100,  1.0000,  1.0000,  1.0000,  0.9100,  0.9100, 

1.0000,  }, 

{88.0000,  9.4000,  1.0000,  1.1500,  0.9400, 

1.1500,  1.3500,  1.2100,  1.0000,  0.8700,  1.0000, 

1.0000,  1.0000,  1.0000,  1.0000,  0.8200,  1.1000, 

1.0800,  }, 

{  7.3000,  2.1400,  1.0000,  0.8800,  1.0000, 

1.0000,  1.0000,  1.0000,  1.0000,  1.0000,  1.1000, 

1.2900,  0.8600,  1.0000,  1.0000,  0.9100,  0.9100, 

1.2300,  }, 

{  5.9000,  1.9800,  1.0000,  0.8800,  1.0000, 

1.0000,  1.0000,  1.0000,  1.0000,  1.0000,  1.0000, 

1.2900,  0.8600,  1.0000,  1.0000,  0.9100,  0.9100, 

1.2300,  }. 

{ 47.0000,  60.0000,  0.5600,  0.8800,  0.9400, 

0.7000,  1.0000,  1.0600,  1.0000,  1.0000,  0.8600, 

0.8200,  0.8600,  1.0000,  1.0000,  1.0000,  1.0000, 

1.0000,  }, 

{  12.0000,  15.0000,  1.0000,  1.0000,  1.0000, 

1.1500,  1.0000,  1.0000,  0.8700,  0.8700,  0.7100, 

0.9100,  1.0000,  0.9000,  0.9500,  0.8200,  0.9100, 

1.0000,  }, 

{  8.0000,  6.2000,  1.0000,  1.0000,  1.0000, 

1.1500,  1.0000,  1.0000,  0.8700,  1.0000,  0.7100, 

0.8200,  0.7000,  1.0000,  0.9500,  0.9100,  1.1000, 

1.0000,  ), 

{41.0000,  8.2000,  1.0000,  0.8800,  0.9400, 

0.8500,  1.0000,  1.0600,  1.1500,  1.0000,  1.0000, 

1.0000,  1.0000,  1.1000,  1.0700,  1.2400,  1.1000, 

1.0000,  }, 

{ 70.0000,  23.0000,  0.9000,  1.0000,  0.9400, 

1.1500,  1.0600,  1.0600,  1.0000,  0.8700,  1.0000, 

1.0000,  1.0000,  1.0000,  1.0000,  0.9100,  1.0000, 

1.0000,  ), 

{ 57.0000,  6.7000,  1.0000,  1.1500,  0.9400, 

1.3000,  1.1100,  1.0600,  1.0000,  1.0000,  0.8600, 

1.1300,  0.8600,  1.1000,  1.0700,  1.1000,  1.1000, 

1.0800,  }, 

{ 50.0000,  28.0000,  1.0000,  1.0000,  0.9400, 

1.1500,  1.0000,  1.0000,  0.8700,  0.8700,  0.8600, 

1.0000,  0.8600,  0.9000,  1.0000,  0.8200,  1.0000, 

1.0000,  >, 

}; 
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int  vil_fitness_uble_B  -  VAL_FnNESS_WIDTH; 

GENERIC  vaLfitness_Uble[VAL_FITNESS_HElGirn[VAL_FITNESS_WIDTHl; 


Perl  Script  for  Creating  Genetic  Programming  Input  Testing  Data  File 

"val_deftne.pr' 

(Author:  Ranjan  Bagchi) 


#!^ublic/gDu/bin^al  — 

#  define  ~  generates  static  data  instantiating  all  necessary 
d  tenninals. 

Sfirst  ■  0; 

Soutf «  "val_$ARGV[0]"; 

open(OUT.">Sout£c"); 

open(OUT_h,*'>$outfh"); 

While(o)  { 

if(nw*$/)  { 

next; 

} 

@l»spIitC‘); 
ift$firsf“0){ 
print  OUT 

''#inchide  <stdio.h>\n", 

"^include  \''gpc.h\"\n", 

"^include  \"prob.h\"\n", 
iprintf("#include  \''%s.h\''\n'',Soutl), 

"GENERIC 

vaI_fitnessjable_t[VAL_FrrNESS_WIDTH][VAL_FITNESS_HEIGHT]  * 
Sformat  •=  -\t{"."%8.4£;\t"x($#l+l).-),\n"; 

> 

Sfirst ++; 

printfi^OUT  $foTmat,@l); 


) 

piintfi:OUT"};\n"); 

print  OUT  "int  val_fitne8sjlable_sz  *  VAL_FrrNESS_WIDTH;\n*; 
print  OUT  "GENERIC 

val_filness_Uble[VAL_FITNESS_HEIGHTI[VAL_FrrNESS_WID'IHl;"; 
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dose(OUT); 


printi(OUT_h  “#define  VAL_FITNESS_HEIGHT  %d\n*,  $#1+1); 
printl(OUT_h  “#define  VAL.FTTNESS^WIDTH  %d\n",  Sfirst); 
pnnt4[0UTJi  ”\ii\nexteni  GENERIC 

val  ftiiess_uble[VAL  FITNESS_HEIGHTl[VAL_FITNESS_WIDTH);\ii"); 
pnnti(OUT_li  ”\ii\nextem  GENERIC 

vd_toess_ubleJ[VAL_ITn«SSJVroTHl[VAL_FrrNESS_HEIGHTl;\n*); 
close  (OUT_li); 


SGPC  Makefile 

COCOMO  Training  and  Testing  Phase 


SGPC;  Sinqile  Genetic  Progratnining  in  C 

(c)  1993  by  Walter  Alden  Tackett  and  Aviram  Canni 

This  code  and  documentation  is  copyrighted  and  is  not  in  the  public  domain. 

All  rights  reserved. 

•  This  notice  may  not  be  removed  or  altered. 

•  You  may  not  try  to  make  money  by  di^ributmg  the  package  or  by  usmg  the 
process  that  the  code  creates. 

.  You  may  not  distribute  modified  versions  without  clearly  documenting  your 
changes  and  notifying  the  princ^al  author. 

•  The  origm  of  this  software  must  not  be  nnsrepresented,  either  by 
eiqilicit  claim  or  by  onnssion.  Since  few  users  ever  read  sources, 
credits  must  appear  in  the  documentation. 

•  Altered  veraons  must  be  plainly  marked  as  sudi,  and  must  not  be 
misrepresented  as  being  the  original  software.  Since  few  users  ever  read 
sources,  credits  must  appear  in  the  documentation. 

-  Tile  authors  are  not  re^onable  for  the  consequences  of  use  of  this 
software,  no  matter  how  awful,  even  if  they  arise  fi’om  flaws  in  it. 
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## 

##  If  you  make  changes  to  the  code,  or  have  suggestions  for  changes, 
##  let  us  know!  (^c@i]>ld01. hac.com) 


M  Sid:  Makeffle,v  1.1  1993/04/30  05:01:48  ^o>avc  E}q>  gpo>avc  $ 

# 

#  invoke  this  MakdEQe  with  the  command;  make  [  TYP£>type  ] 

# 

#  where  you  have  the  problennqiecific  fitness  and  setup  files  named 

#  fimess.c  and  setup.c 

« 

#  TYPE  should  usually  equal  int  or  float,  but  in  princ^Ie  can  be  anything. 

# 

#SLog:  Makefile,vS 

#  Revision  1.1  1993/04/30  05:01.48  gpc-avc 

#  Initial  revision 

# 

«  Revision  '  7  1993/04/23  01:56:25  gpoavc 


TYPE  «=  float 
DEBUG  »0 


EXF*cotr42 
VAL  EXP-val  cotc21 

FIAGS  *  -O  -DfYPE=$(TYPE)  -DDEBUG=$(DEBUG) 
-DVAL  EXP*\"$(VAL  EXP).h\"  -DEXP*=\"$(EXP).h\" 


SRCS  senq>.c  fimess.c  S(EXP).c  main.c  S(VAL_EXP).c 
INCS  *  prob.h 
OBJS  »  S(SRCS:%.c*%.o) 

EXE  «gpc-l$(PROBL£M) 

##  NOTE:  last  definition  wins  M 
CC«gcc 

CFLAGS-  -g-L./lib 
#CFlAGS«-0-L./h*b 


Bold  type  indicates  problem  q)ecific 
declarations  that  include  the  COCOMO 
training  and  testing  files  mated  by 
define.pl  and  val_define.pl 


CPPFLAGS  =  S(FLAGS) 

UBS  »  -L../Kb  -lgpc$(TYPE)  -hn 
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gcc  cc  debug>cc  debug>gcc  puiify  prof  all:  S(OBJS) 
(cd../lib;\ 

$(MAKE)  7YPE-$(TYPE)  DEBUG-$(DEBUG)  \ 
CC-$(CC)  CFlAGS»“$(CFLAGSr  S@) 

$(CC)  -o  $(£X£)  KOBJS)  S(LIBS) 


dean; 

/bhi/nn  -f  $(EX£)  $(OBJS)  Makefile.bak 

Teal>dean:  dean 
(cd../lib;\ 

S(MAKE)  TYPE=$(TYPE)  DEBUG«$(DEBUG)  \ 
CC=$(CC)  CF1j4GS="S(CFLAGS)"  dean) 


Iwf-s7  -1  -p"-2  -t"  -m  -t8  $(SRCS)  $(INCS)  |  Ipr 
(  cd  ../lib;\ 

S(MAKE)  TyPE=$(TYPE)  DEBUG«$(DEBUG)  \ 
CC»$(CC)  CFLAGS="$(CFLAGS)"  $@) 


•CO  -1  $(SRCS)  $(INCS)  Makefile 
(cd../Ub;\ 

$(MAKE)  TYPE*$(TYPE)  DEBUG=$(DEBUG)  \ 
CC-S(CC)  CF1AGS="$(CFLAGS)"  $@) 


-d  -1  S(SRCS)  $(INCS)  Makefile 
(cd../fib;\ 

$(MAKE)  TYPE«$(TYPE)  DEBUG=$(DEBUG)  \ 
CC=$(CC)  CFLAGS="$(CFLAGS)"  $@) 


fint : 

S(LINT.c)  $(CFLAGS)  S(FLAGS)  $(SRCS) 

(  cd  \ 

$(MAKE)  TyPE=$(TyPE)  DEBUG*S(DEBUG)  \ 
CC*$(CC)  CFLAGS=*$(CFLAGS)"  $@) 


depend: 

makedepend  -  S(CFLAGS)  -  S(FLAGS)  $(SRCS) 
(cd../lib;\ 

$(MAKE)  TYPE=$(TYPE)  DEBUG=$(DEBUG)  \ 
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CC-S(CC)  CFLAGS-"$(CFLAGS)-  S@) 


it  DO  NOT  DELETE  THIS  LINE  —  make  depend  depends  on  it. 

tettq>.o:  /usr^dude/stdio.h  ../fib/^c.h  ../Hb^roto.h  ../fib/random.h 
setup.o:  prob.h 

fitness-o:  Aisr^chide/stdio.h  Aisr/lnchide/malloc.h  Aisr/mchide/enno.h 
fitness.o:  Aisi/inchide/sys/enno.h  ../Ub/gpc.h  ../lib/protoJi 
fitness.o:  ../lib/Tandom.h 
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